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ABSTRACT 



This guide is designed to assist state and local consortia 
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Prep stakeholders in secondary schools and two-year colleges, the guide also 
promotes current planning and future development of Tech Prep education 
programs. Section I provides an overview about evaluating Tech Prep education 
programs; a description of the compliance requirements set by federal 
legislation; definitions of a Tech Prep student, concentrator, and completer; 
model for Tech Prep evaluation; and a discussion of the use of internal and 
external evaluators. Section II reports on the seven essential program 
elements of Tech Prep and provides state and local Tech Prep evaluation 
examples. Section III describes Tech Prep performance measures for reporting 
the Perkins four core indicators, and it provides performance measure 
examples. Section IV provides information to assist with collecting, 
analyzing, and presenting Tech Prep program and student outcomes data. 

Section V links the Tech Prep evaluation process to program improvement. 
Appendixes, amounting to over two- thirds of the guide, include glossary; 40 
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Abstract 

Tech Prep education programs play a vital role in the education of American youth. 
During the past decade, with funding from Perkins legislation, Tech Prep consortia have 
consolidated and developed programs for students. The Carl D. Perkins Vocational and 
Technical Education Act of 1998 requires each state to identify performance levels relevant to 
career and technical education. Seven essential program elements and four core indicators 
provide the foundation to evaluate Tech Prep education programs. The purpose of this guide is to 
assist those responsible for Tech Prep evaluation to report program and student outcomes data, 
and to integrate evaluation results with future Tech Prep education program planning efforts. 
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Introduction 

This guide has been developed to assist state and local consortia with their evaluation of 
Tech Prep education programs and with their reporting of Tech Prep program and student 
outcomes data to meet the accountability requirements of the Perkins Act of 1 998. The guide 
also promotes current planning and future development of Tech Prep education programs. 
Practical evaluation examples have been provided to aid state and local evaluators responsible 
for the planning, data collection, and analysis of Tech Prep education programs. This guide is 
designed for Tech Prep stakeholders in secondary schools and two-year colleges. Stakeholders 
include teachers, administrators, counselors, state and local Tech Prep coordinators, and other 
interested parties. 

Due to the variability of state and local Tech Prep evaluation requirements, consortia have 
considerable flexibility in designing an evaluation system to meet their individual needs. 
Therefore, no single evaluation method is prescribed; state and local consortia may select from a 
variety of suggestions and evaluation methods. Any of the methods presented in this guide can 
be modified to meet the needs of state or local consortia. 

The guide is divided into five sections. Section I provides an overview about evaluating Tech 
Prep education programs; a description of the compliance requirements set by federal legislation; 
definitions of a Tech Prep student, concentrator, and completer; models for Tech Prep 
evaluation; and a discussion of the use of internal and external evaluators. Section II reports on 
the Tech Prep seven essential program elements and provides state and local Tech Prep 
evaluation examples. Section III describes Tech Prep performance measures for reporting the 
Perkins four core indicators, and it provides performance measure examples. Section IV provides 
information to assist with collecting, analyzing, and presenting Tech Prep program and student 
outcomes data. Section V links the Tech Prep evaluation process to program improvement. The 
guide also includes a glossary, references, and appendices. Numerous survey templates and 
evaluation examples are described in Section II and are reproduced in the appendices. In 
addition, Web sites and contact information for additional resources and technical assistance are 
provided. 

The guide has been developed following several on-site interviews with state and local Tech 
Prep coordinators and telephone interviews with local Tech Prep coordinators. Tech Prep Web 
sites were reviewed to obtain examples of Tech Prep surveys and definitions of Tech Prep 
student, concentrator, and completer. 
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SECTION 1: EVALUATING TECH PREP EDUCATION PROGRAMS 

During the past decade, Tech Prep has developed in scope and content to meet the needs of 
students in secondary schools and two-year colleges. Nationally, between 1991 and 1997, more 
than 1,000 Tech Prep consortia were created, covering approximately 70% of secondary school 
districts, and serving about 90% of all American high school students (Hershey, Silverberg, 
Owens, & Hulsey, 1998). A Tech Prep consortium is defined as a group of participants from 
secondary schools and two-year colleges, higher education institutions, employers, and public 
and/or private providers in a community responsible for guiding the planning, implementation, 
and monitoring of Tech Prep. 

The collection, analysis, and use of Tech Prep data at the local, state, and national levels has 
been minimal. Telephone interviews with state and local Tech Prep coordinators indicated that 
time, resources, and the turnover of local Tech Prep coordinators are factors that impact their 
efforts to collect and report Tech Prep data on an annual basis. While recent evaluations of Tech 
Prep education programs show many promising trends and challenges, a study conducted by 
Bragg ( 1 997a) found: 

Of nearly 50% of all local Tech Prep consortia in the United States, 40% reported 
they had not even begun to implement formal evaluations of their Tech Prep 
programs. Another 30% indicated their consortia were in the planning stage of 
evaluation, showing only a minority of Tech Prep consortia were actively 
implementing formal evaluations, and most of these were very preliminary, (p. 7) 

Research indicates that although program implementation has been widespread, the reporting 
of student outcomes is unclear or limited (Bragg, Puckett, Reger, Thomas, Ortman, & Domsife, 
1997b; Silverberg, Hulsey, & Hershey, 1997). A four-year longitudinal study began in January 
1998 to better understand the relationship between Tech Prep implementation and student 
outcomes (Bragg, 2001). Data collected assessed Tech Prep initiatives and how they influenced 
students’ educational experiences and outcomes. 

Most consortia report plans to develop comprehensive student databases, but thus far they 
have not implemented them (Silverberg, et al., 1997). Research conducted by Brown, Pucel, 
Twohig, Semler, and Kuchinke (1998) identified two major problems related to Tech Prep 
evaluation efforts: (a) a lack of specific definitions or criteria to identify a Tech Prep student, and 
(b) a lack of consistent processes to identify a Tech Prep student. A national Tech Prep study 
conducted by Ruhland, Custer, and Stewart (1995) concluded that Tech Prep student 
identification is a critical factor in implementing systemic change in Tech Prep education 
programs. Without a Tech Prep student definition, consortia are unable to identify the data 
required to report and evaluate Tech Prep program and student outcomes. This leads to the 
inability to evaluate and report Tech Prep education program results within secondary schools 
and two-year colleges. 
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Compliance with Federal Legislation 

The evaluation of Tech Prep education programs offers many benefits including the 
compliance with the rules and regulations of the federal legislation. Two key benefits are 
improving programs and providing accountability (Boulmetis & Dutwin, 2000; Connell & 
Mason, 1995; Logan, 1999). 

Perkins Program Improvement Requirements 

Section 204 of the Carl D. Perkins Vocational and Technical Education Act 1998 (American 
Vocational Association, 1998, p. 135) outlines the content for Tech Prep education programs. 
Each consortium receiving Perkins funding is required to submit as part of its state plan a five- 
year plan for the development and implementation of Tech Prep education programs. 

Evaluations of Tech Prep for program improvement purposes will enable state and local 
consortia to identify both strengths and areas for improvement. Program improvement efforts 
can be targeted for short-term (i.e., less than one year) or long-term (i.e., more than a year) 
planning and implementation. 

To assist with program improvement efforts, Tech Prep evaluators should collect program 
outcomes data. State and local consortia should include program outcomes data for each of the 
Tech Prep seven essential program elements: (a) articulation agreement, (b) appropriate 
curriculum design, (c) curriculum development, (d) in-service teacher training, (e) counselor 
training, (f) equal access for special populations, and (g) preparatory services. Processes to report 
program outcomes data for the seven essential program elements are described in greater detail 
in Section II. 



Perkins Accountability Requirements 

Federal legislation requires evaluation of Tech Prep education programs. Under section 113 
of the Carl D. Perkins Vocational and Technical Education Act of 1998, the law states that each 
eligible agency shall identify its state plan for core indicators of performance for vocational and 
technical education that include, at a minimum, measures for each of the following: 

(i) Student attainment of challenging State established academic, and vocational 
and technical skill proficiencies. 

(ii) Student attainment of a secondary school diploma or its recognized 
equivalent, a proficiency credential in conjunction with a secondary school 
diploma, or a postsecondary degree or credential. 

(iii) Placement in, retention in, and completion of, postsecondary education or 
advanced training, placement in military service, or placement or retention in 
employment. 
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(iv) Student participation in and completion of vocational and technical education 
programs that lead to nontraditional training and employment. (American 
Vocational Association, 1998, p. 98) 



For each of the four core indicators, states must establish valid and reliable performance 
measures that specify levels of performance which can at a minimum: 

(I) be expressed in a percentage or numerical form, so as to be objective, 
quantifiable, and measurable, and 

(II) require the State to continually make progress toward improving the 
performance of vocational and technical education students. (American 
Vocational Association, 1998, p. 99) 



For the purposes of accountability, Tech Prep evaluators need to collect student outcomes 
data. Student outcomes are defined as changes that occur in individuals as a result of 
participation in an educational experience (Bragg, 1992). Student outcomes should be assessed 
for each of the Perkins III core indicators: (a) student attainment, (b) credential attainment, (c) 
placement and retention, and (d) participation in and completion of non-traditional programs. 
Examples of student outcomes that have been evaluated in various states include (a) student 
perceptions of the Tech Prep program; (b) placement in a program related to career training; (c) 
retention in employment related to career training; (d) an increase in academic skills; (e) a 
decrease in remediation courses; (f) attainment of an associate degree, baccalaureate degree, or 
postsecondary certificate or credential; (g) transition to a baccalaureate program in a career field; 
(h) enrollment in advanced skill courses; (i) an increase in graduates obtaining high-wage jobs; 
and (j) a decrease in unemployment wages paid to program participants. The core indicators 
along with Tech Prep performance measures will be discussed in Section III. 



Core Indicator Framework 

The Office of Vocational and Adult Education (OVAE, 2000a) provides a framework to 
ensure that state performance measures are sufficiently rigorous. For a copy of the core indicator 
framework, access the Web site at http://www.ed.gov/offices/OVAE/tahome.html . Then locate 
the subject column and the title Accountability Systems Development, January 2000 version of 
the Core Indicator Framework (FY2000-3). 



The core indicator framework identifies the purposes and construction of each core indicator 
and possible approaches to collecting data. Since 1998, eight states have been working with 
OVAE to test the use of the core indicators for secondary education (OVAE, 2000b): Florida, 
Illinois, Indiana, Missouri, New Jersey, Ohio, Texas, and Virginia. Since 2000, 10 states have 
been working with OVAE to test the use of postsecondary core indicators (OVAE, 2000c): 
California, Illinois, Minnesota, Missouri, New Jersey, North Carolina, Ohio, Texas, Utah, and 
Wisconsin. Findings from the secondary and postsecondary pilot projects suggest that most 
states can provide data for the Perkins III core indicators. However, the states report major 
differences in the ways in which they define student populations and collect data. 
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It is important to note that the core indicator framework provides a guideline for all career 
and technical education programs. Because Tech Prep is a subset of career and technical 
education programs, not all of the core indicators may apply. Tech Prep evaluators will need to 
select the core indicators and performance measures that are relevant to their Tech Prep 
education program and align them with any pertinent state Tech Prep efforts. The core indicators 
of performance will require states to report Tech Prep data for secondary and postsecondary 
students. 

A template for completing each core indicator for the four key areas (goals, performance 
measures, measurement approaches, and quality criteria) is provided in Appendix A. This 
framework provides a foundation for reporting specific Tech Prep outcomes for Perkins III 
accountability requirements. 

Performance Measures and Standards Example 

Nebraska is one state that has developed a process to gather Tech Prep education program 
data related to Perkins performance measures. In Nebraska, data collection begins at the 
secondary level with those students entering and remaining in a sequence of courses identified as 
a Tech Prep education program by the consortium articulation agreements. The performance 
measures and standards are identified for learning outcomes and placement outcomes (see 
Appendix B). Currently, state personnel are reviewing the list of Nebraska's measures and 
standards to ensure that they specifically address the Perkins core indicators. Nebraska's 
measures and standards may provide a starting point for state and local Tech Prep consortia in 
identifying performance measures and standards in response to Perkins accountability 
requirements. 



Definition of a Tech Prep Student, Concentrator, and Completer 

Prior to beginning the Tech Prep evaluation process, states need to define a Tech Prep 
student, Tech Prep concentrator, and Tech Prep completer at the secondary and postsecondary 
levels. A state definition will provide consistency and commonality when reporting Tech Prep 
data. The task of defining a Tech Prep student has not been easy, and most states continue to 
struggle with developing a definition. Without state definitions, it is difficult to obtain consistent 
data related to program and student outcomes. 

The National Association of Tech Prep Leadership (NATPL) developed possible definitions 
for Tech Prep secondary and postsecondary students by surveying state Tech Prep coordinators. 
The Executive Committee and Research Committee of NATPL (C. Jurgens, personal 
communication, November 14, 2000) provide the following definitions. 

A Tech Prep secondary student has indicated a Tech Prep career pathway and is enrolled in a 
Tech Prep course of study that: 

• includes a technical component, 

• consists of a minimum of two years secondary and two years of postsecondary 
study, 
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• is carried out under a written articulation agreement, 

• may allow the student to earn postsecondary credit while in secondary school, and 

• leads to a specific postsecondary two-year certificate, degree, technical diploma, 
apprenticeship, or baccalaureate degree. 

A Tech Prep postsecondary student is enrolled in a two-year certificate, degree, technical 
diploma, or apprenticeship program and has participated in a secondary Tech Prep course of 
study that: 

• included a technical component, 

• consisted of a minimum of two years at the secondary level, 

• was carried out under a written articulation agreement, and 

• may have allowed the student to transfer in postsecondary credit earned at the 
secondary school. 

To assist with the reporting of the Perkins core indicators, definitions for a Tech Prep 
concentrator and Tech Prep completer need to be determined. OVAE (2000a) defines a 
vocational concentrator as a “student who enrolled in at least one vocational-technical education 
course” (p. 10). OVAE defines a vocational completer as a: 

Student who attained the academic and technical knowledge/skills/proficiencies 
within a program/sequence of courses or instructional units that provides an 
individual with the academic and technical knowledge/skills/proficiencies to 
prepare the individual for employment and/or further/advanced education, (p. 1 0) 

NATPL defines a Tech Prep completer as a “student who has participated in both the 
secondary and postsecondary portions of the recognized education plan and has received an 
appropriate postsecondary two-year certificate, degree, technical diploma, or apprenticeship 
license” (C. Jurgens, personal communication, November 12, 2000). The NATPL and OVAE 
definitions can serve as a starting point to define a Tech Prep concentrator and Tech Prep 
completer. Definitions developed by several states for a Tech Prep student, Tech Prep 
concentrator, and Tech Prep completer are provided in Appendix C. These definitions were 
obtained from state Tech Prep coordinators and Tech Prep Web sites. 

Models for Tech Prep Program Evaluation 

“An evaluation model not only provides the overall framework for evaluation but also gives 
shape to the research questions, organizes and focuses the evaluation, and informs the process of 
inquiry” (Conrad & Wilson, 1985, p. 19). 

Choosing an Evaluation Model 

There are several evaluation models (Worthen, Sanders, & Fitzpatrick, 1997) that could 
guide Tech Prep program evaluation. The objectives-oriented model (Worthen, et al., 1997) 
determines the extent to which objectives are being achieved. This model has measurable 
objectives and uses instruments to gather data. Data reveal discrepancies between objectives and 
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performance. The management-oriented model (Worthen, et al., 1997) assists with decision- 
making. This model evaluates all stages of program development and is often used for 
accountability. The participant-oriented model (Worthen, et al., 1997) responds to an audience’s 
requirement for information. This model focuses on description and judgment, with emphasis on 
understanding the information collected. Scriven’s formative and summative evaluation model 
(Borg & Gall, 1989) is also used to evaluate education programs. In formative evaluation, 
decisions are made as to where and what improvements in an ongoing program are needed. Local 
Tech Prep consortia that periodically review their goals for each of the Tech Prep essential 
program elements are involved with formative evaluation. Recommendations to revise any goals 
prior to the end of the current year may be based upon improvements needed that were not 
initially identified as Tech Prep program goals. Summative evaluation determines the overall 
effectiveness of a fully developed program. Local Tech Prep consortia that complete an end-of- 
the year report are involved in a summative evaluation. This report often requires each local 
consortium to assess the goals and outcomes achieved. At the state level, the evaluation report 
may or may not be used to determine future funding for each local Tech Prep consortium. 
Formative evaluations may involve a decision to continue, expand, or drop the program. 

Management-oriented Evaluation: A Four-step Approach 

For the purpose of Tech Prep evaluation, the management-oriented evaluation model is the 
most useful. The rationale for using the management-oriented model is that “evaluative 
information is an essential part of good decision making and that the evaluator can be most 
effective by serving administrators, policy makers, boards, practitioners, and others who need 
good evaluative information” (Worthen, et al., 1997, p. 97). The management-oriented 
evaluation model provides information that is useful for making decisions about the reallocation 
of funds. 

In a management-oriented approach, Tech Prep can be evaluated following a four-step 
approach to planning and conducing an effective evaluation (Dutton, Hammons, Hudis, & 
Owens, 1994; Fleishman, 1995; Levesque, Bradby, Rossi, & Teitelbaum, 1998). An overview of 
the four-step approach is provided in Table 1. 
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Table 1 

A Four-step Approach to Planning and Conducting an Evaluation 



STEP 1: 
Identify the 
objectives of the 
evaluation. 


■ identify what needs to be measured (the specific objective) 

■ define the participants 


STEP 2: 

Choose the 
evaluation method. 


■ decide how to measure (measurement approach and data 
sources) 

■ choose what these data will be measured against 


STEP 3: 

Collect the data. 


■ decide who will collect the data 

■ determine how the data will be recorded 

■ specify when the data will be collected (frequency and timeline) 

■ collect both baseline data and ongoing data 


STEP 4: 

Analyze and 
communicate the 
results. 


■ categorize and code data 

■ find meaning and interpret the data 

■ look for trends and underlying causes 

■ summarize the information into key points 

■ make recommendations and plan for the future 



Step 1 : Identify the objectives of the evaluation . The first step of conducting the Tech Prep 
evaluation is to identify what needs to be measured (the specific objectives). This is an 
important step and should begin with specific and clearly defined objectives. After identifying 
what to measure, the participants need to be defined. As this document suggests, participants are 
best defined as Tech Prep students, concentrators, and completers at the secondary school and 
two-year college level. The use of a state definition, if available, will ensure a greater degree of 
consistency among data being reported by consortia within the state. 

Step 2: Choose the evaluation method . To measure achievement of the objectives, the next 
step is to decide on an evaluation method (e.g., tests, surveys, interviews) to obtain data. 
Evaluators need to select an evaluation method that is realistic and achievable, given the existing 
resource and time constraints. According to Judy Larson, Tech Prep Coordinator, St. Louis 
Community College, Missouri (personal communication, September 19, 2000): 

Make evaluation simple but thorough enough that it will not be looked at as a 
threat to what we are doing, but as a means to make what we are doing better. 

When people hear the word evaluation, they are turned off because they think it 
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will be used against them. If evaluation is done properly, it is a means of 
strengthening our endeavors and making us better at what we are doing. 

Once the method is selected, decisions need to be made about the specific data to be 
collected-for example, remediation rates, numbers of articulated credits used at the 
postsecondary level, and numbers of completers of associate degree programs. Existing data 
sources and those that could be developed should be explored. The method selected should 
ensure that data are valid, reliable, cost-effective, systems-focused, and useful to key 
stakeholders. To demonstrate progress or impact, the data need to be compared to a baseline 
(starting point), benchmark, standard (local, state, or nationally defined), cohort, or the same 
group/individual over time. 

Step 3: Collect the data . Based on the evaluation method and potential data sources, the third 
step is to collect the data. Will data be collected by a department, a secondary school, a two-year 
college, a consortium, or an outside agency? What is the role of one's institutional research 
office? Collaborating with others in one's consortium helps to avoid duplication of effort. A 
timeline specifying dates for the beginning and completion of the data collection process is an 
important element of the data collection step. 

Step 4: Analyze and communicate the results . Once the data have been collected, the fourth 
step is to categorize and code the data. Analysis of the data includes looking for trends and 
underlying causes. Are there any areas that show strong results or particularly poor results? Key 
findings should be identified, and recommendations should be presented in a final report which 
is communicated to internal and external stakeholders. 

To further assist with implementing the four steps to planning and conducting an evaluation, 
OVAE (2000a) has identified five general quality criteria for performance measurement as part 
of the Perkins core indicator framework. “These general quality criteria are intended to insure 
that states have chosen a measurement approach that has sufficient rigor and comparability” (p. 
8). Evaluators should consider these criteria when implementing the four steps to planning and 
conducting an evaluation. The five criteria are: 

1. Validity-the degree to which the performance measurement approach directly and fully 
measures the student outcomes at an appropriate time interval. Performance measurement 
approaches produce valid data when they use assessment and other data collection 
instruments that have strong content validity; that is, they directly measure what they are 
supposed to measure. Indirect or proxy measures are valid to the extent that they are 
associated with or highly predictive of student outcomes. In addition, performance 
measures are valid when measurement is conducted at the appropriate times, that is, 
concurrent with services or after services, so that reasonable inferences can be made 
about the effectiveness of career and technical education. 
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2. Reliability-the degree to which performance measurement is conducted in a consistent 
manner using standardized or consistent data collection instruments (e.g., student record 
forms, surveys, assessment instruments) and effective management information systems 
to insure data quality. Performance measures are reliable when repeated measurements 
yield similar results and when measurement is conducted consistently across student 
groups and entities. 

3. Cost-effectiveness-the degree to which the performance measurement uses measurement 
approaches and data collection systems that provide the highest-quality data at the lowest 
possible costs. Performance measurement is most cost-effective when states make the 
fullest possible use of existing data systems and when they share data systems with other 
programs. 

4. System focused-the degree to which states develop common or consistent measurement 
approaches and data collection systems with other programs within education and 
workforce development systems. 

5. Management utility — the degree to which measurement approaches are useful in managing 
continuous improvement at the state and local levels. Management utility is highest when 
performance measurement approaches are easy to understand and use and provide timely 
data to users to facilitate continuous improvement. 

Use of Internal and External Evaluators 

Formal evaluation has traditionally been seen as the responsibility of a select few researchers 
and academics. Increasingly, evaluators are recognizing that practitioners often bring the most 
meaningful and useful information to the evaluation process and the analysis of data (Hitchcock 
& Hughes, 1995). Key stakeholders also should be involved in the evaluation process from the 
beginning. 

State and local consortia will need to decide whether internal or external evaluators will 
conduct the evaluation. Internal evaluators are employees of the organization that design and 
conduct the program evaluation. If a local consortium forms a Tech Prep evaluation committee, 
it may assist with determining the Tech Prep education program needs and identifying data 
collection methods for state and local reports. Specific goals of a Tech Prep evaluation 
committee might include (a) development of an evaluation design for program effectiveness, (b) 
development of data collection instruments and processes, (c) development of a timeline, (d) 
collection and analysis of data, (e) preparation of an evaluation report communicating findings to 
local and state personnel, and (f) reevaluation of the program design and recommendation for 
improvements or changes. Most importantly, the committee can review evaluation data to assist 
in future program planning. 

Carole Swinehart, Tech Prep Administrative Resource Teacher, Hillsborough Tech Prep 
Consortium, Florida (personal communication, June 20, 2000) stated: 
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As a result of state leadership, we developed our evaluation committee; we 
worked on the evaluation plan . . . and what we needed to improve. The 
evaluation plan is like the marketing plan . . . shows progress . . .serves as a 
baseline, or it helps as a foundation to start improving our program. 

External evaluators provide professional advice but are not permanent employees of the 
organization (Worthen, et al., 1997). External evaluators are often third-party evaluators, 
independent evaluators, consultants, university research personnel, or state agencies. There are 
distinct differences between internal and external evaluators. Table 2 summarizes some of the 
advantages and disadvantages of using internal or external evaluators. States that have used 
internal or external evaluators to assist with their Tech Prep evaluations are listed in parentheses. 
State and local consortia will need to decide which evaluators will assist with evaluation efforts. 

Table 2 

Advantages and Disadvantages of Internal and External Evaluators 



Evaluators 


Advantages 


Disadvantages 


Internal Evaluators (CA, FL, 
MN, NC, OR) 


• Possess first-hand knowledge of 
the education industry. 

• Understand the culture of the 
school or consortium. 

• Have strong working 
relationships with practitioners 
and stakeholders, which may 
facilitate the evaluation process. 

• Are often able to conduct long- 
term projects due to constant on- 
site access. 


• May have limited 
knowledge and/or 
experience of evaluation 
and broad systemic issues. 

• May be questioned as to 
their credibility due to 
conflict of interest. 

• May have limited time and 
availability for evaluation 
activities because of other 
responsibilities in their 
institution. 


External Evaluators (FL, IL, 
OH, OR, TX) 


• Bring objectivity and a fresh 
perspective to the evaluation. 

• Possess experience or skills that 
internal evaluators may lack. 

• Can offer the time and stay 
focused on the evaluation. 

• Can cross-pollinate approaches 
used by similar clients. 

• May elicit franker comments 
from individuals. 

• Possess in-depth knowledge of 
standards across the state. 


• Need to spend time 
developing trust and 
credibility within the 
organization. 

• May not understand the 
culture of the school or 
consortium. 

• Must rely on the 
cooperation and active 
participation of internal 
employees. 

• Can be expensive; 
therefore input may be 
limited by funding. 
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Summary 

Section I provided an overview for evaluating Tech Prep education programs and compliance 
with federal legislation. Tech Prep education programs are evaluated to improve programs and 
provide accountability. It is important to reiterate that when evaluating Tech Prep education 
programs, both program and student outcomes data should be collected. Several evaluation 
models exist to guide the Tech Prep evaluation process. A four-step approach to planning and 
conducing an evaluation was presented in Section I. Section II will report on the Tech Prep 
seven essential program elements and present state and local evaluation models to assist with 
Tech Prep evaluation efforts. 
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SECTION II: REPORTING ON THE TECH PREP 
SEVEN ESSENTIAL PROGRAM ELEMENTS 

Each consortium receiving Perkins funding is required to submit as part of its state plan a 
five-year plan for the development and implementation of Tech Prep education programs. These 
plans are expected to report on the seven essential program elements required of Tech Prep 
education programs. The Perkins seven essential program elements include: (a) articulation 
agreement, (b) appropriate curriculum design, (c) curriculum development, (d) in-service teacher 
training, (e) counselor training, (f) equal access for special populations, and (g) preparatory 
services. 

In this section, the essential elements are described, and an overview of the 1998 Perkins Act 
requirements and specific evaluation questions to collect program and student outcomes data are 
provided for each program element. 



Articulation Agreement 

The Perkins Act (Section 204) requires that each Tech Prep education program shall “be 
carried out under an articulation agreement between the participants in the consortium” 
(American Vocational Association, 1998, p. 136). Each consortium should sign agreements that 
commit to articulated programs with “a non-duplicative sequence of classes and other 
experiences providing progressive achievement leading to competencies in a Tech Prep 
education program” (Brustein, 1993, p. 14). Career pathways have been developed to ensure that 
students have opportunities to explore careers and to connect career and technical education 1 and 
academic instruction. These pathways provide students with a career and technical education 
program of study extending from secondary schools to two-year colleges 2 and allow students to 
earn college credit while completing work in high school. 

Oregon is an example of a state that has developed career pathway strategies that include (a) 
arts and communications continuing to a two-year college offering degrees in journalism and 
visual communications; (b) business and management offering degrees in accounting, business 
technology, computer science, and hospitality and tourism management; (c) health services with 
degree programs in nursing assistant, associate degree nursing, and practical nursing; (d) human 
services with degree programs in early childhood education, education, fire protection 
technology, and human services; (e) industrial and engineering with degrees in drafting 
technology, computer-aided design, electronics technologies, and manufacturing engineering 
technologies; and (f) natural resource with degrees in agriculture and forestry (Mid-Willamette 
Education Consortium, 1 999). 

Some states have developed one articulation agreement for all schools in the consortium (i.e., 
all pertinent courses in both secondary and postsecondary institutions), and others have 
developed individual articulation agreements for each course within the consortium. The state of 



'During the past decade, there has been a national trend to use the term career and technical education for 
vocational and technical education. This guide uses career and technical education (CTE). 

" For the purpose of this guide, two-year colleges include technical and community colleges. 
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Minnesota requires that an articulation agreement for two-year colleges possess a philosophical 
framework (allowing each college to frame its institutional agreement to reflect consistency with 
its educational philosophy), have a commitment to develop a 2 + 2 Tech Prep education 
program, provide standards for advanced standing agreements, and have the signatures of 
executive officers of the participating institutions (M. Messimer, personal communication, July 
20, 2000). Further, Minnesota has developed a review process to assist local consortia with their 
articulation agreements. This process requires that articulation agreements for advanced standing 
be reviewed in a timely fashion to assure continued commitment and course integrity. The 
timeline identified allows consortia with a large number of articulation agreements to adjust the 
review process when there is a change in faculty or administration at either the secondary or 
postsecondary level. 

Examples of articulation agreements are in Appendix D. When assessing articulation 
agreements, evaluators should answer the following questions: 

What process is used for developing articulation agreements? 

Was the consortium articulation agreement developed with stakeholder input? 

- What types of written articulation agreements are in place between secondary schools and 
two- and four-year institutions? 

What evidence exists that articulation agreements reflect a minimum of a 2 + 2 program 
of study for each Tech Prep career pathway? 

What evidence exists that articulation agreements are being used? 

What is the process to evaluate articulation agreements? 

Appropriate Curriculum Design 

Section 204 of the Perkins Act (American Vocational Association, 1998) requires that each 
Tech Prep education program shall: 

Consist of at least 2 years of secondary school preceding graduation and 2 years 
or more of higher education, or an apprenticeship program of at least 2 years 
following secondary instruction, with a common core of required proficiency in 
mathematics, science, reading, writing, communications, and technologies 
designed to lead to an associate's degree or a postsecondary certificate in a 
specific career field, (p. 136) 

When assessing curriculum, design evaluators should ask the following questions: 

How is the Tech Prep model structured (e.g., 2 + 2, 2 + 2+ 2, or 4 + 2)? 
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What is the percent increase in the number of students completing a two-year college 
program within three years of initial entry compared to the previous year's baseline data? 

What is the percent increase in the number of high school students completing courses 
that earn advanced standing credit at a two-year college compared to the previous year's 
baseline data? 

What is the percent increase in overall student academic achievement as a result of Tech 
Prep education programs? 

How is the curriculum designed to ensure a common core of required proficiencies in 
mathematics, science, reading, writing, communications, and technologies that leads to 
an associate's degree or two-year certificate in a specific career field? 

What evidence indicates an increase in the number of students who participate in 
academic and career and technical education integrated curriculum? 

What is the process to evaluate curriculum design? 

Curriculum Development 

Section 204 of the Perkins Act (American Vocational Association, 1998) requires that each 
Tech Prep education program include the development of Tech Prep curricula for both secondary 
and postsecondary participants in the consortium that: 

(A) meet academic standards developed by the state; 

(B) link secondary schools and two-year postsecondary institutions, and if 
possible and practicable, four-year institutions of higher education through 
nonduplicative sequences of courses in career fields, including the 
investigation of opportunities for Tech Prep secondary students to enroll 
concurrently in secondary and postsecondary coursework; 

(C) use, if appropriate and available, work-based or work site learning in 
conjunction with business and all aspects of an industry; and 

(D) use educational technology and distance learning, as appropriate, to involve 
all the consortium partners more fully in the development and operation of 
programs, (p. 136) 

Questions to ask when assessing Tech Prep curriculum development include: 

What staff development opportunities are provided for teachers to learn about and use 
curriculum integration? 
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Is there a decrease in the number of students in remedial courses who enroll in a two-year 
college program the semester following high school graduation compared to the previous 
year's baseline data? 

Is there an increase in the percentage of students enrolled in work-based learning 
experiences linked to industry skills standards and state-issued skill certificates compared 
to the previous year's baseline data? 

How are secondary faculty and two- and four-year college faculty working together to 
plan, develop, and implement a Tech Prep education program of study? 

Is there a curriculum development team? If so, who are the team members? 

How are career exploration and planning courses made available to students? 

In-service Teacher Training 

Section 204 of the Perkins Act (American Vocational Association, 1998) requires that each 
Tech Prep education program include in-service training for teachers that: 

(A) is designed to train vocational and technical teachers to effectively implement 
Tech Prep programs; 

(B) provides for joint training for teachers in the Tech Prep consortium; 

(C) is designed to ensure that teachers and administrators stay current with the 
needs, expectations, and methods of business and all aspects of an industry; 

(D) focuses on training postsecondary education faculty in the use of contextual 
and applied curricula and instruction; and 

(E) provides training in the use and application of technology, (p. 136) 

Questions to ask when assessing in-service teacher training include: 

What types of staff development activities are provided? 

What have teachers learned as a result of participating in staff development activities? 

How do new or substantially revised academic courses emphasize contextual learning? 

Which secondary academic and career and technical teachers have received staff 
development to reinforce mathematics, science, and communications competencies as an 
integral part of career and technical education? 

Which two-year college academic and career and technical teachers have received staff 
development activities? 
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What internship opportunities are provided to expose teachers and administrators to 
industry work sites and labor force expectations? 

- Have consortium stakeholders attended and participated in local, state, and national 
conferences? 

What is the process to evaluate in-service teacher training activities? 

Counselor Training 

Section 204 of the Perkins Act (American Vocational Association, 1998) requires that Tech 
Prep education programs include training for counselors designed to enable them to more 
effectively: 

(A) provide information to students regarding Tech Prep education programs; 

(B) support student progress in completing Tech Prep programs; 

(C) provide information on related employment opportunities; 

(D) ensure that such students are placed in appropriate employment; and 

(E) stay current with the needs, expectations, and methods of business and all 
aspects of an industry, (p. 137) 

Questions to ask when assessing counselor training include: 

What staff development activities have been provided to counselors to assist with the 
counseling and advising of Tech Prep students in secondary schools and two-year 
colleges? 

Do students in grades 9 to 12 prepare a written career plan that outlines high school work 
and/or high school to two- or four-year education plans leading to future employment 
goals? 

Is there an increase in the number of students enrolling in a two-year college the semester 
following high school graduation compared to the previous year's enrollment in two-year 
colleges? 

What types of counselor internship opportunities are available for high school counselors 
to leam about two-year college programs? 

What are the indicators to show an increase in Tech Prep awareness among high school 
and two-year college counselors? 

What is the process to evaluate activities designed for counselor training? 



National Research Center for Career and Technical Education 



19 



Measuring Tech Prep Excellence: A Practitioner's Guide to Evaluation 



Equal Access for Special Populations 

Section 204 of the Perkins Act requires that each Tech Prep education program “provide 
equal access to the full range of technical preparation programs to individuals who are members 
of special populations, including the development of tech-prep program services appropriate to 
the needs of special populations” (American Vocational Association, 1998, p. 137). Special 
populations are defined as: 

(A) individuals with disabilities; 

(B) individuals from economically disadvantaged families, including foster children; 

(C) individuals preparing for nontraditional training and employment; 

(D) single parents, including single pregnant women; 

(E) displaced homemakers; and 

(F) individuals with other barriers to educational achievement, including 
individuals with limited English proficiency, (p. 90) 

Questions to ask when assessing equal access for special populations include: 

How do promotional items for Tech Prep marketing reflect educational equity for special 
populations? 

How is Tech Prep serving special populations? 

Describe the Tech Prep experiences that have benefited special populations in secondary 
schools and two-year colleges. 

How are services provided to allow equal access for special populations? 

Preparatory Services 

Section 204 of the Perkins Act requires that each education program “provide for preparatory 
services that assist participants in tech-prep programs” (American Vocational Association, 1998, 
p. 137). Preparatory services include outreach to potential career and technical education 
students, career and personal counseling, and vocational assessment and testing. Preparatory 
services are provided to students not yet enrolled in Tech Prep. The delivery of services is before 
the 1 1 th grade. Questions to ask when assessing preparatory services include: 

What types of services does the consortium provide to assist students in secondary 
schools in the selection of or preparation for appropriate Tech Prep education program of 
study? 
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Does the consortium have a Tech Prep marketing plan? How is the marketing plan 
implemented and evaluated? 

What are the promotional activities for students in grades 8 to 12, parents, 
businesspeople, and community members? 

What is the process to evaluate preparatory services? 

Additional Tech Prep Program Elements 

The Perkins Act does not limit Tech Prep evaluation to the Tech Prep seven essential 
program elements. Some states have developed additional program elements to increase the 
breadth or depth of the Tech Prep education program. One element is work-based learning 
experience, which allows students to understand all aspects of the industry and connect school- 
based learning with work-based learning. Experiences may include job shadowing, mentoring, 
internships, cooperative education, or youth apprenticeships. The work-based learning 
experience is closely related to the student's course work and pertinent to the contents of the 
school curriculum. Criteria for evaluating these programs include the number of businesses that 
provide work-based learning, the types of experiences available, and the number of work-based 
learning sites related to the student's career and technical education course work. Questions to 
ask when assessing work-based learning include: 

Does the work-based learning experience pertain to the student's career pathway/major 
choice? 

Does the work-based learning experience include criteria for participating? 

Has work-based learning been infused into the Tech Prep education program? 

What is the student's perception regarding the benefit of the work-based learning 
experience? 

What is the percent of Tech Prep students participating in a work- based learning 
experience? 

Another potential program element is business, industry, and community involvement. 
Collaboration with business, industry, and community partners may be driven by the need for 
qualified graduates in the workplace. Criteria for evaluating this program element include an 
employer's degree of satisfaction with the student, involvement of partners with curriculum 
planning, and overall participation of partners in Tech Prep activities. Questions to ask when 
assessing business, industry, and community involvement include: 



- What businesses sponsor Tech Prep student scholarships? 

To what extent have business, industry, and community members designed programs to 
orient teachers to workplace skills? 
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Do business, industry, and community members view their involvement as beneficial to 
students? 



State Evaluation Examples 

Some states have developed statewide Tech Prep evaluation models that provide a 
framework for evaluating Tech Prep for accountability and program improvement purposes. The 
examples that follow provide options to collect both program and student outcomes data. State 
and local consortia can modify any one of the examples to assist with their evaluation efforts. 

Connecticut's Indicators 



Connecticut has developed the Tech Prep Success Analysis and Measurement Indicators 3 for 
state Tech Prep secondary and postsecondary participants. The 1 1-item indicator analysis centers 
on various Tech Prep components. The analysis uses quantitative responses related to program 
and student outcomes. The indicators cover articulation agreements, 2 + 2 program design, 
student diversity, employer satisfaction, and student participation, completion, and employment. 
The 1 1 indicators are as follows: 

1 . Number of students enrolled in Tech Prep at secondary schools with which the 
community-technical college has articulation agreements. 

2. Secondary schools partnering with community-technical colleges in Tech Prep 
agreements. 

3. Diversity of secondary school Tech Prep students 

4. Secondary school programs (those that have Tech Prep courses in areas of 
communications, math, science, and career) for which Tech Prep agreements exist and 
the number enrolled in each program. 

5. Community-technical college programs for which outcome statements have been 
developed and aligned with Tech Prep outcomes and Connecticut Employer Developed 
Skills Standards. 

6. The availability of transition services. 

7. Number of high school graduates who entered community-technical college who were 
enrolled in secondary Tech Prep through any community-technical college agreement at 
any time in any secondary school. 

8. Number of graduates who were enrolled in Tech Prep in secondary school. 

9. Number of college graduates who were enrolled in Tech Prep in secondary school who 
attain employment in their field of study within six months of graduation. 

3 Permission to reprint material obtained from the Tech Prep Director, Connecticut State Department of Education. 
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10. Number of employers of college graduates (who were enrolled in Tech Prep in secondary 
school) who indicate that they are satisfied with skills presented by community-technical 
college graduates. 

11. Number of Tech Prep students who enrolled in a community-technical college and have 
transferred to a baccalaureate granting institution, with or without an Associate Degree. 

Florida's Tech Prep Consortia Annual Reporting 

Florida's Tech Prep Consortia Annual Report has been designed to enhance the quality, 
effectiveness, and achievement of Tech Prep goals for each consortium. Beginning January 1, 
1993, Florida International University was granted a project entitled Performance-Based Project 
for the Development of a Florida Statewide Plan for Evaluation. The project's activities included 
the planning, development, and implementation of a statewide plan to evaluate Tech Prep 
activities. The review included collecting data to assist with the preparation of an annual report 
from each consortium. The annual report provides information on a consortium's 
accomplishments and identifies measurable benchmarks that can be used for future comparisons. 
Florida's Tech Prep Consortia 4 suggest that Tech Prep should be evaluated to: 

1. Document the extent to which each of Florida's Tech Prep consortia is attaining its 
goals/objectives as stated in its proposal to the Florida Department of Education. 

2. Collect information for determining the characteristics of each of Florida's Tech Prep 
consortia, including (1) number and characteristics of students served, and (2) 
accomplishments and progress toward implementing each of the federally required key 
elements for Tech Prep. 

3. Collect data to determine the impact of Tech Prep by identifying measurable benchmarks 
and parameters that can be used for programmatic comparison in the future. 

4. Collect information to determine the status of Florida's role in supporting its local Tech 
Prep consortia and providing needed technical assistance. 

A copy of the survey used to collect data for Florida's Tech Prep Consortia Annual Report is 
included in Appendix E. 

Illinois' Tech Prep Evaluation System 

In Illinois, prior to 1998, evaluation of Tech Prep education programs was carried out by 
local consortia and the Illinois State Board of Education (ISBE), but these activities typically 
addressed a distinct aspect of Tech Prep rather than an entire program. To address this void, the 
Tech Prep Evaluation System for Illinois (TPESI) was developed through an initiative involving 
the Office of Community College Research and Leadership (OCCRL) at the University of 
Illinois at Urbana-Champaign (UIUC), the ISBE, and the Illinois Community College Board 



4 Permission to reprint material obtained from F. Hammons, Associate Professor, Florida International University. 
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(1CCB). Goals that guide the TPESI system and provide a rationale for Tech Prep evaluation 
include: 

Describe the status of Tech Prep implementation in Illinois. 

Identify participants in Tech Prep and describe how the participation of various Tech 
Prep student groups changes over time. 

Identify the benefits (outcomes) of Tech Prep for students, especially outcomes linked to 
student learning. 

Identify the benefits (outcomes) of Tech Prep for other stakeholder groups. 

Discern strategies that support the continuous improvement of Tech Prep within consortia 
statewide and at the state level. 

State agencies specify that funds for Tech Prep be used to support the Tech Prep seven 
essential program elements. Illinois has added an eighth program element-work-based learning 
experiences. A copy of the TPESI is included in Appendix F. For each school and college site 
visit, team members rate the implementation stage and quality of the eight Tech Prep essential 
elements and the eight Tech Prep supporting elements as part of the School Assessment Form. 
The Consortium Assessment Form assesses each essential element and supporting element for 
the consortium overall. This form includes (a) stage of implementation, (b) quality of element, 
and (c) additional comments and recommendations. A matrix is provided for each element that 
includes a description for program and student outcome measures. 

Texas Site-based Peer Review Process 

Texas recently developed a site-based peer review process to assess each consortium in a 
range of areas and subareas. Review areas include program, instruction, counseling, professional 
development, marketing, budgeting, planning, student success, and evaluation. For each subarea, 
criteria are provided along with measurement statements, core standard descriptions, and 
recommended resources. Reviewers must assess whether the consortium does not meet, meets, or 
exceeds the standard, and they must provide explanatory comments. A complete copy of the self- 
study, peer review process, and evaluation form is included in Appendix G. An example of one 
review area is shown below. 

Review Area : Evaluation 
Subarea : Student Follow-up 

Description of Criteria : The consortium has a method for the identification and follow-up 
of Tech Prep students in both secondary and postsecondary programs. 

Resources : Data from multiple systems, college procedures for awarding articulated 
credit, and college procedures for identifying Tech Prep students. 

Measure 1 Consortium can identify secondary Tech Prep students. 

Core Standard : Consortium member secondary schools have a method to identify a 
student who has completed the high school portion of the Tech Prep program. 

Measure 2 Consortium can identify postsecondary Tech Prep students. 



24 



National Research Center for Career and Technical Education 



Measuring Tech Prep Excellence: A Practitioner's Guide to Evaluation 



Core Standard : Consortium member colleges have a method to identify a student who is 
matriculating from a secondary Tech Prep program and to record the award of college 
credit for classes taken in high school, where applicable. 

Reviewer Summary : (does not meet, meets, or exceeds standards) 

Comments : (from reviewers) 

West Virginia’s “STARS” 

West Virginia's Tech Prep standards are based on 20 “STARS” (Strategies That Advance 
Reform in West Virginia Schools) that cover areas including curricula, stakeholder support, 
marketing, and assessment measures. Each of the STARS identifies specific performance 
concepts, associated documentation data, and suggested strategies to achieve STARS standards. 
The documentation data section provides a list of items that consortium members can review to 
identify standards. All of the STARS have been compared with the National Association Tech 
Prep Leadership's (NATPL) quality indicators and Perkins III core indicators. The STARS are 
rated based on the presence or absence of STARS documentation data. 

Consortia conduct a self-assessment of the STARS each year and submit the findings to the 
state Tech Prep director. Once every three years, an on-site technical review by the state Tech 
Prep director and a team of local Tech Prep coordinators follows this self-assessment. (A 
complete copy of the STARS document can be obtained by contacting West Virginia's Tech Prep 
Director at 304-558-241 1.) Two examples of West Virginia's STARS follow. 5 

STAR 1 1 (Stakeholder Support): Counselors are actively involved in the Tech Prep 
initiative. Counselors play an important role in assisting students with career decisions for life 
after school. It is critical that counselors are well informed about the many opportunities 
available to students who choose the Tech Prep curricula. 

Documentation Data: Three of the four following items must exist: 

1. Counselor's council or cabinets exist within the Tech Prep consortia and meet on a 
regular basis focusing on Tech Prep issues and concerns. 

2. Counselors are knowledgeable of Tech Prep options and career offerings through 
attendance of workshops, individual research, Internet training, and so on. 

3. Counselors provide technical assistance for advisor/advisee programs concerning the 
skilled/Tech Prep component within the programs of study. 

4. Counselors demonstrate knowledge of applied academics through proper student 
placement in applied courses. 



5 Permission to reprint material obtained from West Virginia's Tech Prep Director. 
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Suggested activities specified in the STARS documentation include: 



Develop counselor councils that focus on career information and Tech Prep issues. 

Tech Prep coordinator participates in county counselors' meetings to provide career 
information or other information counselors may need. 

The consortium provides professional development that targets career information, five- 
year plans, work-based learning, and Tech Prep initiatives. 



STAR 20 (Assessment): School systems utilize data to make decisions. The collection of data 
must be followed by analysis of the data, using it to make changes in areas of low performance 
or to enhance successful practices. Assessments need to be made on an annual basis to sustain a 
quality Tech Prep initiative. Consortia must collect, analyze, and apply the knowledge learned to 
improve Tech Prep. 



Documentation Data: 



1 . Define the process being used to analyze the collected data. 

2. Report the findings and the changes made within the Tech Prep initiative that are a direct 
result of the data. 

Suggested activity specified in the STARS documentation include: 

The consortium conducts an annual workshop at the end of each school year to examine 
data from the Tech Prep initiative and set future goals and objectives based on the 
findings. 

Wisconsin's Performance Measures 



Wisconsin has developed Tech Prep performance measures that are incorporated into 
existing local data systems. The performance measures focus on particular elements of the Tech 
Prep education program and are quantitative in nature. Student data are reported through the 
Vocational Education Enrollment Report System (VEERS) managed by the Department of 
Public Instruction and the Client Reporting System at the Wisconsin Technical College System 
Board. Local consortia use the 1 5 performance measures when reporting student and program 
outcomes at the end of each year. Most consortia have been collecting and reporting these data 
for the past five years. This has allowed local consortia to set baseline data and identify goals for 
several years. Wisconsin's 1 5 performance measures are: 6 

1. The number of high school graduates enrolling in technical college the semester 
following high school graduation. 



6 



Permission to reprint material obtained from the Executive Director, Governor's Work-Based Learning Board. 
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2. The number of high school graduates entering a one- or two-year technical college 
program the semester following high school graduation who need remedial course work 
in order to be admitted to or to receive a diploma/degree from the technical college 
program of their choice. 

3. The number of high school graduates completing a one- or two-year technical college 
program (associate degree or vocational diploma/certificate) within three years of initial 
entry. 

4. The number of students in 9 th through 12 th grade having written career plans that outline a 
high school to work and/or high school to post secondary education plan leading to future 
employment goals. The ultimate expectation is that all 1 1 th and 12 th grade students will 
have these plans. 

5. The rate of participation by 1 1 th and 12 th graders in selected Tech Prep activities by 
gender will exceed 25%. 

6. The rate of participation of 1 1 th and 12 th grade special populations in selected Tech Prep 
activities will be greater than or equal to the rate of participation of 1 1 th and 12 th grade 
general population students in the same activities. 

7. The number of 9-12 secondary students in school-supervised, work-based learning 
experiences. 

8. The number of high school students in 1 1 th and 12 th grade who successfully completed 
course work that would be eligible for advanced standing credit. 

9. The number of high school graduates who applied advanced standing credit to a technical 
college program after high school. 

10. The number of high school students in 1 1 th and 12 th grade who earned transcripted credit 
(refers to that credit granted by the technical college system for technical college 
courses). 

11. The number of high school graduates who applied transcripted credits to a technical 
college program after high school. 

12. The number of high school courses that are revised and upgraded to include integrated 
and applied learning activities. 

13. (a) The number of secondary teachers, counselors, principals, assistant principals, and 
directors of instruction in internships that result in improved curriculum, teaching 
strategies, or counseling strategies. 

(b) The number of postsecondary teachers and administrators in internships that result in 
improved curriculum and teaching strategies. 
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14. The number of high schools with curriculum maps developed for technical college 
associate degree programs and two years of a baccalaureate degree. 

15. The number of secondary schools that included Tech Prep in district wide educational 
policy statements. 



Local Evaluation Examples 

Some consortia have developed local evaluation models to collect data unique to their 
Tech Prep programs. Secondary schools and two-year colleges can use these examples to guide 
the development of their own evaluation and reporting plans. 

The Central Montana Tech Prep Consortium uses a Likert six-point rating scale to help 
secondary schools and two-year colleges self-assess their progress on a range of factors. The 
factors evaluated include commitment of leaders to Tech Prep, faculty and staff awareness of and 
involvement in Tech Prep, the existence of articulation agreements, provisions for professional 
development, a plan for career guidance, relevance of instruction to the labor market, appropriate 
curriculum design, organization of administration, clearly developed marketing plan, outreach 
program for target populations, system to monitor data collection results, and a system for annual 
review. A complete copy of Central Montana's Assessment of Tech Prep Components document 
is provided in Appendix H. 

In addition to the state and local evaluation examples provided, a rubric has been developed 
to assess Tech Prep program and student outcomes (see Appendix I.) The rubric provides 
components that a local consortium should evaluate. Consortia rate each component using the 
criteria provided. Following the assessment of each component, a consortium should develop an 
action plan to improve the areas where no effort or minimal efforts were reported. 

Summary 

This section described the Tech Prep seven essential program elements and provided 
questions to assess each program element. Selected state and local Tech Prep evaluation models 
were presented to provide a framework for evaluating Tech Prep education programs. State and 
local consortia can modify any one of the models to meet their evaluation efforts. Section III will 
discuss the Perkins core indictors and provide Tech Prep performance measure examples for the 
1 5 subindicators within each core indicator. 
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SECTION ID: REPORTING ON TECH PREP PERFORMANCE MEASURES 
FOR PERKINS CORE INDICATORS 

The Office of Vocational and Adult Education (OVAE) has developed a core indicator 
framework to assist with the Perkins IIII requirements for performance reporting. As previously 
discussed in Section I, each state plan must identify performance measures for the core indicators 
(OVAE, 2000a). The core indicator framework is for all career and technical education (CTE) 
programs. 



Overview of Perkins Core Indicators 

The Perkins core indicators include (a) student attainment, (b) credential attainment, (c) 
placement and retention, and (d) participation in and completion of nontraditional programs. 

First, each core indicator has been divided into secondary subindicators (denoted by S in the 
subindicator title) and postsecondary subindicators (denoted by P in the subindicator title). 
Second, each core indicator has been divided into subindicators for measurement. For example, 
core indicator one, student attainment, has been divided into the subindicators academic 
attainment and vocational and technical skill attainment. Each of the core indicators and 
subindicators can be used as a starting point to develop Tech Prep performance measures at the 
secondary and postsecondary levels. 

Tech Prep Performance Measures 

Each state is responsible for identifying and collecting data related to each performance 
measure. A performance measure is defined as “the type of outcome that is considered 
appropriate for monitoring” (Hoachlander, Levesque, & Rahn, 1992, p. 9). This section provides 
examples of Tech Prep performance measures to assist with data collection in response to the 
Perkins core indicators. These examples demonstrate different approaches to measuring the 15 
subindicators. Each statement specifies (a) a quantifiable measure (e.g., percentage, number), (b) 
a timeline (e.g., two semesters, six months, one year), and (c) the sample population (e.g., Tech 
Prep student, completer, or concentrator). The targets (e.g., 3% improvement) can be set against 
benchmarks, standards, or previous performance. Data from each of these performance measures 
can be analyzed, summarized, and submitted as part of the state plan for vocational education 
and the reporting on Perkins III accountability requirements. 

Evaluators should plan to develop performance measures that show the impact of Tech Prep 
on student outcomes. Examples of performance measures include the number of articulated 
credits applied toward two- and four-year degrees, the number of skill-enhanced courses taken at 
the two-year college level, the percentage of students requiring remediation, the number of 
students obtaining employment directly related to their two-year college degree, standardized test 
score gains, competency or skills attainment, course completion rates, drop-out rates, graduation 
rates, and employment and unemployment rates. Evaluators should also use baseline data for 
comparison and ensure that Tech Prep students, concentrators, and completers are defined. The 
Perkins performance measures below were modified to address Tech Prep performance 
measures. The following examples were derived from Perkins III state plans: 
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Core Indicator 1: Student Attainment 
1S1: Secondary Academic Attainment 

Subindicator goal: All Tech Prep 7 students who reach a state-defined 
threshold level of career and technical 8 education will master academic 
knowledge and skills that meet challenging state and academic standards. 

Sample Tech Prep Performance Measures: 

a) percent of Tech Prep concentrators will complete the high school 

graduation requirements. 

b) There will be a percent increase of graduating Tech Prep 

concentrators scoring at or above the national average on a standardized 
test. 

c) The number of Tech Prep students enrolled in advanced placement courses 

will increase by percent. 

1S2: Secondary Career 9 and Technical Skill Attainment 
Subindicator goal: All Tech Prep students who reach a state-defined 
threshold level of career and technical education will master the knowledge 
and skills that meet state-established, industry-validated career and technical 
standards. 

Sample Tech Prep Performance Measures: 

a) The percentage of secondary Tech Prep concentrators who meet or exceed 

the minimum competency requirements measured by state-administered, 
industry- validated instruments that are valid and reliable will increase by 
percent by (insert year). 

b) The percent of Tech Prep students who complete an approved career and 
technical education (CTE) program and receive a “C” or better GPA in the 

CTE program will increase percent in comparison with a 

corresponding average of the previous two years. 

c) The Tech Prep student score on a licensure or certification examination, 

for those fields in which licensure or certification is required, industry- 
endorsed competency examination, or a state-recognized test will increase 
by percent by (insert year). 



7 For the purpose of this guide, the subindicator goals have been defined for Tech Prep. The data collected for the 
subindicator goals should relate to Tech Prep students in secondary schools and two-year colleges. 

For the purpose of this guide, career and technical has replaced the word vocational. 

For the purpose of this guide, career has replaced the word vocational. 
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1P1: Postsecondary Academic Attainment 

Subindicator goal: All Tech Prep students who reach a state-defined level of 
career and technical education to complete a postsecondary program will 
master the knowledge and skills that meet program-defined academic 
standards. 

Sample Tech Prep Performance Measures: 

a) percent of matriculated postsecondary Tech Prep students who 

enrolled in the fall of each year in academic and career and technical 
courses will successfully complete the courses as measured by credits 
earned at the end of the semester. 

b) percent of postsecondary Tech Prep concentrators will receive a “C” 

or better in general education courses at the postsecondary level. 

c) The number of students entering postsecondary education requiring 

remediation will decrease by percent the year following high school 

graduation. 

1P2: Postsecondary Career and Technical Skill Attainment 
Subindicator goal: All Tech Prep students who reach a state-defined 
threshold level of career and technical education to complete a postsecondary 
program will attain the knowledge and skills that meet program-defined and 
industry-validated career and technical standards. 

Sample Tech Prep Performance Measures: 

a) On average, there will be a percent increase in Tech Prep students 

enrolling in credits beyond their postsecondary degree requirements. 

b) percent of postsecondary Tech Prep graduates of degree programs 

with professional licensure exams who take the exams will pass. 

c) percent of postsecondary Tech Prep students will have attained a 

degree, a certificate, apprenticeship, or industry certification two years 
following enrollment in the degree program. 

d) percent of postsecondary Tech Prep completers will attain a 3.0 GPA 

on a 4.0 scale in career and technical courses each semester following 
enrollment in the degree program. 
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Core Indicator 2: Credential Attainment 
2S1: Secondary Completion 

Subindicator goal: All Tech Prep students who reach a state-defined 
threshold level of career and technical education will attain a secondary 
school diploma or its recognized state equivalent. 

Sample Tech Prep Performance Measures: 

a) percent of Tech Prep students who graduated will obtain a high school 

diploma or its recognized equivalent each year. 

b) The rate at which secondary Tech Prep concentrators become completers 

will be percent for (insert year) and percent over four years. 

2S2: Proficiency Credential with Secondary Diploma 
Subindicator goal: All Tech Prep students who reach a state-defined 
threshold level of career and technical education will attain a proficiency 
credential in conjunction with a secondary school diploma or its state- 
recognized equivalent. 

Sample Tech Prep Performance Measures: 

a) percent of Tech Prep students who graduate with a high school 

diploma will equal or exceed the statewide graduation rate each year. 

b) percent of Tech Prep concentrators will receive a high school diploma 

or a high school equivalency and will receive a local, state, or national 
credential, and exit secondary education during the reporting year. 

c) The percentage of secondary Tech Prep completers who receive a 

proficiency credential will increase by percent by (insert year). 

2P1: Postsecondary Degree or Credential 

Subindicator goal: All Tech Prep students who reach a state-defined 
threshold level of career and technical education to receive postsecondary 
degrees, certificates, or credentials will receive these postsecondary 
credentials. 

Sample Tech Prep Performance Measures: 

a) percent of the postsecondary Tech Prep students will obtain an 

associate degree or technical certificate within three years of enrolling in 
the degree program. 
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b) The rate at which postsecondary Tech Prep concentrators become 

completers will be percent for (insert year) and percent 

over four years. 

c) percent of postsecondary Tech Prep students will successfully pass a 

state and/or national licensure/certification examination in 

(insert name of program). 

Core Indicator 3: Placement and Retention 

3S1: Secondary Placement 

Subindicator goal: All Tech Prep students who reach a state-defined 
threshold level of career and technical education during secondary education 
will successfully transition to postsecondary education or advanced training, 
employment, and/or military service. 

Sample Tech Prep Performance Measures: 

a) percent of Tech Prep students will be placed in postsecondary * 

education or advanced training within months of high school 

graduation. 

b) Within one year of high school graduation, at least percent of Tech 

Prep concentrators will matriculate into a postsecondary education 
program or registered apprenticeship. 

c) percent of secondary Tech Prep completers will be enrolled in a 

postsecondary education program within six months of graduating from 
high school. 

3S2: Secondary Retention 

Subindicator goal: All Tech Prep students who reach a state-defined 
threshold level of career and technical education and who leave secondary 
schooling and who are placed in postsecondary education or advanced 
training, employment and/or military service will be retained in these 
activities. 

Sample Tech Prep Performance Measures: 

a) percent of Tech Prep concentrators who respond to the follow-up 

survey will still be engaged in postsecondary education and/or 
employment within one year of graduation. 
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b) percent of Tech Prep students who completed an approved Career 

Technical Education program left school and were placed in 
postsecondary education or employment six months after graduation. 

3P1 : Postsecondary Placement 

Subindicator goal: All Tech Prep students who reach a state-defined 
threshold level of career and technical education to complete a postsecondary 
program will successfully transition to further education or advanced 
training, employment, and/or military service. 

Sample Tech Prep Performance Measures: 

a) The number of Tech Prep students who obtained employment directly 

related to their postsecondary degree has increased by percent. 

b) percent of postsecondary Tech Prep completers responding to the 

graduate survey will be in baccalaureate programs or employed (insert 

number) months after high school graduation. 

c) percent of postsecondary Tech Prep completers will be employed in 

positions related to their program of study or will have continued their 
education (insert number) months after high school graduation. 

3P2: Postsecondary Retention 

Subindicator goal: All Tech Prep students who reach a state-defined 
threshold level of career and technical education who leave secondary 
schooling and who are placed in further postsecondary education or 
advanced training, employment, and/or military service will be retained in 
these activities. 

Sample Tech Prep Performance Measures: 

a) percent of postsecondary Tech Prep completers who respond to the 

graduate survey will be retained in baccalaureate programs or employed 
(insert number) months after high school graduation. 

b) percent of postsecondary Tech Prep completers will be employed 

within (insert number) months of graduation and continue to be 

employed 6 months following initial employment. 

c) percent of postsecondary completers (two-year) articulated credits to a 

four-year institution and are pursuing baccalaureate degrees. 
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Core Indicator 4: Participation in and Completion of Nontraditional 
Programs 

4S1: Participation in Secondary Nontraditional Programs 
Subindicator goal: All secondary career and technical education programs 
preparing Tech Prep students for further training and employment in 
careers or occupations with significant underrepresentation of males or 
females will increase participation of underrepresented males or females. 

Sample Tech Prep Performance Measures: 

a) The number of Tech Prep students who enrolled in and completed a 
nontraditional career and technical education program within industry 
clusters will be 



b) The rate at which secondary Tech Prep concentrators access programs 
preparing for training and employment in careers or occupations with 

significant underrepresentation by gender will be percent for 

(insert year) and percent over four years. 

c) At least percent of Tech Prep students in underrepresented gender 

groups will be enrolled in courses that have been identified as leading to 
nontraditional employment for that gender. 

4S2: Completion of Secondary Nontraditional Programs 
Subindicator goal: All secondary career and technical education programs 
preparing Tech Prep students for further training and employment in 
careers or occupations with significant underrepresentation of males or 
females will increase program completion for underrepresented males or 
females. 

Sample Tech Prep Performance Measures: 

a) The number of nontraditional Tech Prep students who enrolled in and 
completed a career and technical education program within industry 
clusters will be 



b) percent of secondary Tech Prep completers in a nontraditional career 

and technical education program will be from underrepresented gender 
groups. 

c) The rate at which secondary Tech Prep concentrators complete programs 
preparing for further training and employment in careers or occupations 

with significant underrepresentation by gender will be percent in 

(insert year) and percent over four years. 
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d) The percentage of Tech Prep concentrators in underrepresented genders in 
nontraditional programs who do not complete the nontraditional program 
will decrease by percent by (insert year). 

4P1: Participation in Postsecondary Nontraditional Programs 
Subindicator goal: All postsecondary career and technical education 
programs preparing Tech Prep students for further training and 
employment in careers or occupations with significant underrepresentation 
of males or females will increase participation of underrepresented males or 
females. 

Sample Tech Prep Performance Measures: 

a) percent of postsecondary Tech Prep students participating in a 

nontraditional career and technical education program will be from 
underrepresented gender groups. 

b) The rate at which postsecondary Tech Prep concentrators access programs 
preparing students for training and employment in careers or occupations 

with significant underrepresentation by gender will be percent for 

(insert year) and percent over four years. 

4P2: Completion of Postsecondary Nontraditional Programs 
Subindicator goal: All postsecondary career and technical education 
programs preparing Tech Prep students for further training and 
employment in careers or occupations with significant underrepresentation 
of males or females will increase program completion for underrepresented 
males or females. 

Sample Tech Prep Performance Measures: 

a) The percentage of postsecondary Tech Prep students by gender graduating 

from nontraditional degree programs during the most recent academic year 
will increase by percent. 

b) percent of postsecondary Tech Prep completers in a nontraditional 

career and technical education program will be from underrepresented 
gender groups. 

c) There will be a percent increase in total postsecondary Tech Prep 

completers who are from underrepresented genders in nontraditional 
occupational programs from (insert year) to (insert year). 
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Summary 

This section provided an overview of the Perkins four core indicators and 15 secondary and 
postsecondary subindicators. Examples of Tech Prep performance measures were provided to 
assist state and local consortia with data collection. Data collection for each of the subindicators 
will assist with reporting the impact of Tech Prep efforts. The impact will address both program 
and student outcomes data. Section IV describes ways to collect, analyze, and present Tech Prep 
data for the Tech Prep seven essential program elements and performance measures. 
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SECTION IV: COLLECTING, ANALYZING, AND PRESENTING 

TECH PREP DATA 

In Section II, questions were presented to assist with the evaluation of the Tech Prep seven 
essential program elements. In Section III, Tech Prep performance measures were provided to 
report student outcomes data for the Perkins core indicators. This section will assist state and 
local consortia with collecting, analyzing, and presenting Tech Prep student and program 
outcomes data. 

Collecting Data 

Data collection is the process of collecting information for reporting purposes. A timeline for 
completing the evaluation needs to be developed and should anticipate potential hindrances (e.g., 
staffing shortages, program changes). The specific dates for data collection should be determined 
in light of the evaluation objectives and methods. For example, if the objective is to measure 
postsecondary retention over one year, data collection should be designed around this timeline. 

Most evaluators use a combination of data collection methods to obtain the required 
information and assist with documenting impact. According to Ruhland et al. (1995), “given the 
complexity and variety of Tech Prep programs, it is critical that multiple methods be used to 
conduct program assessment” (p. 19). There are multiple methods available to collect data. To 
select the best methods for a particular situation, evaluators should begin with the evaluation 
objectives and then “select the methods that will provide the most credible answers” (McMillan, 
2000, p. 252). A description of three common data collection methods follows. 

Tests 



Tests, commercially produced or independently developed, are frequently used to collect 
data. When selecting or designing tests, evaluators should seek information about the validity 
and reliability of the instruments and how the scores are interpreted. Norm-referenced and 
criterion-referenced tests can help the evaluator compare individuals' scores with scores of other 
groups. A norm-referenced test compares an individual score to the scores of a group of 
individuals using the same test (e.g., Tech Prep students scored above 90 % of all students in the 
computerized accounting class). A criterion-referenced test specifies a specific goal or criterion 
for students to achieve (e.g., Tech Prep completer achieves at least 80 out of 100 on the 
principles of accounting exam). 

To promote validity and reliability, uniform methods for administering and scoring the test 
are specified (McMillan, 2000). To evaluate a commercially developed test, evaluators should 
seek information about the instrument from the developer, industry journals, publishers' test 
catalogs, test critiques, and test directories (Wolf, 1990). Standardized tests appropriate for Tech 
Prep reporting of student academic achievement include ACT, ASSET, and two-year college 
admission tests. Tests used to measure occupational knowledge include Work Keys and licensure 
exams required of certain occupations (e.g., automotive technician, network administrator). 
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Interviews 



Interviews vary from structured question-and-answer formats to informal conversations. 
Although time-consuming, individual or small-group interviews can provide highly detailed and 
useful information for interpretation (Wolf, 1990). Interviewers should be trained to conduct the 
interview and collect data in a consistent manner to ensure that the information is comparable. 
The interviewer's skills in developing rapport, asking appropriate questions, and listening 
effectively are crucial to the success of this data collection method. Examples of questions to ask 
Tech Prep students include: What have been the benefits to participating in Tech Prep? In what 
ways could Tech Prep be improved? What aspects of Tech Prep were most helpful? 

Surveys 



A Tech Prep student identifier survey can be developed to record student information. This 
survey could ask for (a) name, (b), career clusters, (c) course name, (d) gender, (e) social security 
number, (f) date of birth, and (g) plans for first year out of high school. The survey can be in an 
electronic format to simplify data analysis. The following statement can be included on the Tech 
Prep identifier survey to obtain permission to access the student's data: “You are not required to 
provide the following information requested. If you do not wish to respond to an item, leave it 
blank. By signing and dating the survey, you are indicating that you have provided this 
information and give (insert school name) permission to use the data.” The Statement of 
Informed Consent allows states to access and report student information once permission has 
been obtained. Accessing student information at the secondary and postsecondary level will 
assist with identifying Tech Prep students and reporting student outcomes data. 

A number of consortia and states have generously provided copies of surveys they use to 
collect Tech Prep data. These examples serve as a model to assist consortia with the data 
collection process. Administrator, teacher, and counselor surveys are provided in Appendices J, 
K, and L, respectively. These surveys collect information about individuals' involvement in Tech 
Prep education program elements, use of different teaching methodologies, and perceptions of 
Tech Prep. Examples of student follow-up surveys are provided in Appendix M. This example 
can assist consortia in collecting data after students have exited from the Tech Prep education 
program. A parent survey, provided in Appendix N, gathers information about parents' 
understanding and perceptions of Tech Prep. An employer follow-up survey, provided in 
Appendix O, gathers feedback from employers about Tech Prep graduates. 

These sample surveys provide a starting point for state and local consortia designing Tech 
Prep data collection tools. (The author of this guide recommends that you contact the author or 
consortium listed for specific information on the reliability and validity of any of the surveys 
provided.) Thomas (1999) offers the following guidelines for developing and administering 
surveys: 



1 . Plan - Use your objectives to plan the purpose and content of the survey, and 
select the target audience. Determine the resources and personnel required, and 
specify a time frame. 
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2. Develop - Create the survey tool, using questions that probe for specific data. 
Consider the methods that participants will use to respond to questions (e.g., 
yes/no, rating scales, checklists, ranking, open-ended questions). Ensure that clear 
directions are provided with the survey. 

3. Obtain respondents - Seek informed consent from the participants or from their 
parents or guardians before administering the survey. Attach a cover letter to 
explain the purpose of the survey and provide incentives for completing the task. 

4. Prepare for data collection - Pilot test the survey on a sample group to test the 
clarity of instructions and questions, and the validity of the data collected. Design 
a method for tracking the information. 

5. Collect data - Distribute the survey. Collect and record the data from completed 
surveys. Ensure that confidentiality is maintained for all respondents. 

The Center on Occupational Research and Development (CORD) has developed a series of 
surveys to assist local Tech Prep consortia. Five surveys make up CORD's Measures of Tech 
Prep Excellence evaluation. (For additional information, contact CORD at www.cdrd.org) . 
Surveys have been developed to rate local consortia at the secondary and postsecondary levels, 
to measure parent and employer support, and to obtain student feedback. A series of statements 
for each survey are rated on a scale of 0 to 4 to measure the success of the school's Tech Prep 
education program. Examples include the following statements 10 : 

1 . Administrators/leaders in my school are supported by a Tech Prep committee or 
council that improves and sustains our Tech Prep education program. 

2. Counselors in my school are aware of what components and initiatives support 
Tech Prep education programs. 

3. Teachers in my school inform students about options available to them other than 
four-year degrees. 

4. Parents are invited to participate in Tech Prep orientation meetings. 

5. The community or technical college partnering with my school works with 
employers to identify employability standards required by their businesses and 
industries. 

6. Employers partnering with my school actively support Tech Prep education 
programs. 

Improving survey response rates . Thomas (1999) suggests several techniques for improving 
survey response rates. The format of the survey should be appealing to the participant and easy 



10 Permission to reprint material obtained from D. Bond, Vice President for Education/Employer Partnerships, 
CORD. 
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to read. The shorter the survey, the better the response rate will be. The survey should be 
associated with a respected sponsor and prefaced by a cover letter explaining the value of the 
survey data to the person and the education community. Follow-up can be conducted more than 
once, using multiple avenues, such as letters, telephone calls, and e-mail. The survey should also 
be designed to facilitate ease of return for the respondent. Evaluators can include self-addressed 
stamped envelopes, provide e-mail reply addresses, or develop Web-based forms for participants 
to submit. 

Tech Prep evaluators may begin the survey process by educating the participants about the 
purpose and value of the survey well in advance. For example, in the year preceding graduation. 
Tech Prep students from secondary schools and two-year colleges should be informed about 
surveys that will be sent to them after graduation. Students should be encouraged to complete 
and return the surveys to assist with Tech Prep data collection efforts and future program 
planning. 

Tech Prep participant and non-participant survey. On a national level, a survey has been 
developed to obtain information from Tech Prep participants and non-participants (Bragg, Dare, 
Reger, Ovaice, Zamani, Layton, Domsife, Valle, Brown, & Orr, 1999). The survey collected 
data from consortia in eight states and included approximately 4,700 students. A 32-item 
questionnaire (see Appendix P) was devised to collect information about (a) high school 
educational experiences and work, (b) transition to college and college experiences, (c) post-high 
school work experience and expectations, and (d) demographics and personal characteristics. 
Data were collected between 1998 and 1999. This survey can be modified to obtain Tech Prep 
participant and non-participant student data. Findings from the eight states are available in the 
final report (Bragg, et al., 1999). 



Career Clusters 

In addition to collecting data related to a consortium's Tech Prep evaluation objectives, the 
U.S. Department of Education (1999) Consolidated Annual Performance, Accountability, and 
Financial Status Report requires states to collect and report Tech Prep student enrollment data by 
the 16 career clusters. “The significance of the 16 career clusters is that the standards are being 
set at the federal level, but states decide if they want to use the DOE clusters. States may decide 
to adopt all 16 clusters or to develop crosswalks and use their own clusters. They may currently 
have five clusters and decide to break them into 16” (Robertson, 2000, p. 32). The 16 career 
clusters (see Appendix Q) represent careers from entry level through professional and technical 
management. Also included in Appendix R is a list of career clusters and occupations identified 
by OVAE. Career clusters provide students the flexibility to move within clusters and explore 
careers based on their goals and interest. The OVAE career clusters Web site 
(' www.ed.gov/offices/OVAE/clusters) contains individual career clusters, a list of pilot sites, and 
career cluster models. 

“During this transition from the previous classification reporting system to this new career 
clusters reporting system, the decision to place state specific courses and programs in a career 
cluster rests with the state” (U. S. Department of Education, 1999, p. 16a). Many states are 
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moving in this direction. For example, in Missouri efforts have begun to match each career and 
technical education course in secondary schools and two-year colleges with one of the 16 career 
clusters. “The transition to the 16 career clusters will take time but will provide students more 
career options and broaden the existing career avenues. Eventually Missouri will develop a 
crosswalk linking the 16 career clusters with the six career paths presently used in Missouri to 
guide appropriate students through the career exploration process” (H. Kujath, Coordinator, 
Community and Career Education, Department of Elementary and Secondary Education, 
Missouri, personal communication, September 19, 2000). To access Missouri's 16 Career 
Cluster Assignment List, go to the Division of Vocational and Adult Education home page 
http://www.dese.state.mo.us/diwoced/ . Click on Perkins III Accountability Measures - Report 
Forms (top of the page) and then the 16-career cluster list. 

Analyzing Data 

“Data analysis is “the process of making sense out of one's data” (Merriam, 1988, p. 127). 
Analysis is synthesizing and interpreting information in order to develop reasonable conclusions. 
Institutions often collect data but seldom take the next step to analyze the data. “Using data to 
manage and improve programs, academic or vocational, is relatively foreign to many educators 
and administrators” (Hoachlander, Levesque, & Rahn, 1992, p. 1 1 1). If data have been collected, 
often those responsible for program evaluation are not sure how to go about using the data. 

A state or local consortium will need to determine the best way to analyze Tech Prep data in 
light of its evaluation objectives. Database programs provide a useful way for Tech Prep 
evaluators to store, analyze, and manage information. Databases can be used to organize 
particular data elements relating to Tech Prep students, including demographics, enrollment 
details, academic performance, course completion, articulated credits gained, and work-based 
experiences. Databases can summarize the information and create reports based on queries. 

A number of consortia currently use database programs to collect and analyze Tech Prep 
data. Some consortia have access to state-administered databases that manage a broad range of 
student information and provide regular reports. In some cases, consortia have modified these 
databases to meet their unique needs. Some states use commercially developed database 
programs, and other consortia have built their own databases to meet specialized needs. 

Most evaluators report that implementing the use of a database program is expensive and 
time-consuming. Also, there are ongoing costs associated with maintaining and upgrading 
database programs. In the long term, however, the information gained is valuable, and database 
programs provide an efficient way of recording vast amounts of information. State and local 
consortia should look for opportunities to collaborate in collecting and analyzing data. 
Collaborative systems can provide centralized comprehensive information storage, reduce 
individual costs, and promote the exchange of useful data. Tech Prep evaluators should plan 
ways to obtain the resources needed to develop and maintain database programs. A list of local 
consortia that have developed Tech Prep database programs is provided in Appendix R. 
Individuals listed within these consortia can be contacted to discuss the processes used to 
develop their respective database program. 
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Data analysis can occur in a formative manner (ongoing) or a summative manner (at the end 
of the program), depending on the evaluation method used (Dutton et al., 1994). During and after 
the data collection phase, the data are coded as a step in the data analysis process. Coding may 
entail dividing the data by demographics, flagging any points of interest, categorizing according 
to program features, or filing under sub-topics. The analysis should include inspecting the data 
for trends and underlying causes. Are there any areas that show particularly strong results or 
particularly poor results? Do the data change over time? Are there better results at any point 
during the timeline of the data collection? Is it possible that intervening variables might have 
influenced the validity or reliability of the data? 

To enhance the data analysis process, Tech Prep evaluators should become familiar with 
other relevant data, such as state and local data, annual averages, standards, benchmarks, and 
past performance. After the data are analyzed, key points that relate to the initial evaluation 
objectives should be identified. It is appropriate to comment on both strengths and weaknesses of 
the Tech Prep education program being evaluated. At the least, evaluators need to report 
findings as specified by the state plan and as determined to be useful for future planning and 
decision making. 



Presenting Data 

To synthesize and report Tech Prep data, evaluators can use various approaches. Schmoker 
(1996) notes that “data make the invisible visible, revealing strengths and weaknesses that are 
easily concealed” (p. 38). Visual representations of data, such as tables, bar graphs, and pie 
charts, can help present results clearly so that trends and patterns can be identified. Tables report 
the relationship between two or more variables and the frequencies of responses for each 
variable. Bar graphs present the data based upon the number of observations identified for each 
category. Pie charts usually illustrate the proportions or percentages of the total number possible. 
Descriptive summaries can be used to further explain the data and factors involved. Following 
are examples of ways to present Tech Prep data. 



Remediation Rates for Students in Postsecondary Tech Prep 

Programs 


Tech Prep 
program 


Number of 
graduates enrolled 
in postsecondary 
education 


Number and percentage of 
graduates enrolled in 
remedial courses at the 
postsecondary level 


Allied Health 


13 


3 (23%) 


Automotive 


8 


0 (0%) 


Computer 

Support 

Technology 


25 


2 (8%) 
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Articulated Credit Courses at Sample Tech Prep Consortium 


Name of course 


Secondary articulated 
courses (course number) 


Postsecondary articulated 
courses (course name) 


Business Computer 
Applications 


100, 105, 106, 107 


Principles of Computing 
Introduction to MS- 
Windows 


Criminal Justice I 
Criminal Justice II 


100, 104, 106, 107, 110 


Crime in America 
Fundamentals of Criminal 
Law 


Electronics I 
Electronics II 


101, 104, 106, 107, 110 


DC Circuits 
AC Circuits 



Percentage of Secondary Tech Prep Students Retained in 
Employment, Education &/or Military Two Years Post 
Graduation 




Tech Prep Student Scores on Standardized Test 




■ Student Scores in 

1999 

■ Student Scores in 

2000 
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The following example further illustrates how a consortium can analyze and present Tech 
Prep data. 11 These data were collected as part of a national study titled Tech Prep 
Implementation and Preliminary Student Outcomes for Eight Local Tech Prep Consortia 
directed by Bragg et al. (1999) to study Tech Prep implementation and student outcomes. The 
purpose of the study was to determine the degree to which students in the study group (compared 
to the other students) identified as Tech Prep (a) participate in and complete the career and 
technology high school portion of Tech Prep education program (six-year plans approved by the 
state process); and (b) complete more than the state minimum mathematics and/or science 
courses required for high school graduation. The study involved field visits, surveys of Tech 
Prep and non-Tech Prep participants, and analysis of secondary and postsecondary transcripts for 
students in both groups. 

Transcripts for 575 students who graduated in 1995, 1996, and 1997 from six high schools 
were examined to determine the degree to which the students' course-taking patterns aligned with 
state-approved Tech Prep six-year plans. (In the example, data are presented for one of the six 
high schools.) Student transcripts were examined for participation in one or more career and 
technology courses that are part of state-approved Tech-Prep programs (one or more courses 
may be articulated for college credit). The number of career and technology credits completed in 
non-Tech-Prep (TP) programs was also calculated. Data are presented using a variety of tables 
and bar graphs. The examples further illustrate ways to present Tech Prep data. For each 
transcript, the following variables were addressed to assess the career/technology and 
math/science course taking patterns of students: 

1. The number and percentage of TP and non-TP students in each school completing 2 
or more credits (1 credit = 1 year) in the same discipline in more than one state- 
approved Tech-Prep area (Figure 1 and Table 3). 



11 Permission to reprint material obtained from C. Brown (unpublished paper). Director, Texas Statewide Tech Prep 
Leadership and Evaluation. 
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2. The number and percentage of TP and non-TP students in each school completing up 
to 1 and 1/2 career and technology credits that are part of a state-approved Tech-Prep 
program (TP program participant) (Figure 2 and Table 4). 

3. The number and percentage of TP and non-TP students in each school completing 2 
career and technology credits that are part of a state-approved Tech-Prep program (TP 
program concentrator) (Figure 2 and Table 4). 

4. The number and percentage of TP and non-TP students in each school completing 2 
1/2 or more career and technology credits that are part of a state-approved Tech-Prep 
program (TP program completer) (Figure 2 and Table 4). 

5. The number and percentage of TP and non-TP students in each school completing 
more than the minimum mathematics requirements for high school graduation (at 
least 3 credits including Algebra 1, Algebra II, and Geometry) and science (at least 2 
1/2 credits regular-level courses - Biology, Physical Science, Chemistry, Physics, 
Principles of Technology, etc.) (Figure 3 and Table 5). 

6. The number and percentage of TP and non-TP students in each school who qualify as 
a career and technology concentrator (completing 2 or more credits in at least one 
Tech-Prep area) (Figure 4 and Table 6). 

7. The number and percentage of TP and non-TP students in each school who qualify as 
a career and technology concentrator (completing 2 or more credits in at least one 
Tech-Prep area) and who complete more than the minimum state mathematics and/or 
science graduation requirements (Figure 5 and Table 7). 



RESULTS - 110 total student transcripts analyzed 



School year 


Number of Tech-Prep students 


Number of non-Tech-Prep students 


1994-95 


3 


3 


1995-96 


20 


20 


1996-97 


32 


32 
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1994-95 1995-96 1996-97 



Figure 1. Percentage of students completing more than 2 credits in more than one state-approved 
Tech-Prep program 



School year 


1994-95 


1995-96 


1996-97 


Tech-Prep 


1 of 3 (33%) 


3 of 20 (15%) 


5 of 32 (16%) 


Non-Tech-Prep 


1 of 3 (33%) 


4 of 20 (20%) 


3 of 32 (9%) 



Table 3. Number of students completing more than 2 credits in more than one state-approved 
Tech-Prep program 
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Participant Concentrator Participant 



□ Tech-Prep DNon Tech-Prep 

Figure 2. Percentage of students classified as participants, concentrators, or completers of state- 
approved Tech-Prep programs 



School year 


1994-95 


1995-96 


1996-97 


Tech-Prep 

■ Participant 

■ Concentrator 

■ Completer 


1 of 3 (33%) 
Oof 3(0%) 

2 of 3 (67%) 


6 of 20 (30%) 
6 of 20 (30%) 
8 of 20 (40%) 


6 of 32 (19%) 
15 of 32 (47%) 
11 of 32 (34%) 


Non-Tech-Prep 

■ Participant 

■ Concentrator 

■ Completer 

■ Non-participant 


Oof 3(0%) 

2 of 3 (67%) 
1 of 3 (33%) 
Oof 3 (0%) 


7 of 20 (35%) 
9 of 20 (45%) 
2 of 20 (10%) 
2 of 20 (10%) 


15 of 32 (47%) 
7 of 32 (22%) 
4 of 32 (13%) 
6 of 32 (19%) 



Table 4. Number of students classified as participants, concentrators, or completers of state- 
approved Tech-Prep programs 
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B Tech-Prep DNon Tech-Prep 

Figure 3. Percentage of students exceeding the minimum state mathematics and science 
graduation requirements 



School year 


1994-95 


1995-96 


1996-97 


Tech-Prep 

■ Math 

■ Science 


1 of 3 (33%) 
1 of 3 (33%) 


13 of 20 (65%) 
5 of 20 (25%) 


16 of 32 (50%) 
13 of 32 (41%) 


Non-Tech-Prep 

■ Math 

■ Science 


Oof 3(0%) 

2 of 3 (67%) 


12 of 20 (60%) 
12 of 20 (60%) 


12 of 32 (38%) 
11 of 32 (34%) 



Table 5. Number of students exceeding the state minimum mathematics and science graduation 
requirements 
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Figure 4. Percentage of students completing at least 2 credits in a state-approved Tech-Prep 
program {CT concentrator or completer) 



School year 


1994-95 


1995-96 


1996-97 


Tech-Prep 


2 of 3 (67%) 


14 of 20 (70%) 


26 of 32 (81%) 


Non-Tech-Prep 


3 of 3 (100%) 


11 of 20 (55%) 


1 1 of 32 (34%) 



Table 6. Percentage students completing at least 2 credits in a state-approved Tech-Prep program 
{CT concentrator or completer) 




Figure 5. Percentage of students completing at least 2 credits in a state-approved Tech-Prep 
program {CT concentrator or completer) AND completing more than the state minimum math 
and/or science graduation requirement 
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School year 


1994-95 


1995-96 


1996-97 


Tech-Prep 


1 of 3 (33%) 


9 of 20 (45%) 


16 of 32 (50%) 


Non-Tech-Prep 


2 of 3 (67%) 


9 of 20 (45%) 


6 of 32 (19%) 



Table 7. Number of students completing at least 2 credits in a state-approved Tech-Prep program 
(CT concentrator or completer) AND completing more than the state minimum math and/or 
science graduation requirement 



Summary 

Section IV provided information to assist local and state consortia in their efforts to collect, 
analyze, and present Tech Prep data for the purposes of program improvement and program 
accountability. Also included in this section are several methods for presenting the data. Section 
V will link the Tech Prep evaluation process to program improvement and will conclude with 
suggestions for communicating Tech Prep evaluation results to stakeholders. 



52 



National Research Center for Career and Technical Education 



Measuring Tech Prep Excellence: A Practitioner's Guide to Evaluation 



SECTION V: LINKING THE TECH PREP EVALUATION PROCESS 
TO PROGRAM IMPROVEMENT 

An important purpose of conducting a Tech Prep evaluation is to enhance program 
improvement. Evaluation processes should be integrated into Tech Prep education program 
planning so that the results of evaluations can be used to guide decision-making and future 
planning. Ultimately, this should lead to action involving “program change, innovation or 
improvement” (Barak & Breier, 1990, p. 58). 

In this section, four strategies are introduced to link evaluation findings with program 
improvement. The strategies include action-planning, continuous quality improvement, total 
quality management, and Malcolm Baldrige Strategies. By integrating one of these strategies 
with the evaluation process, the institution can plan a process to implement recommendations for 
Tech Prep program improvement. This section concludes with providing suggestions to 
communicate evaluation results. 

Action-planning 

Evaluators can employ action-planning techniques to link the evaluation findings to future 
changes in Tech Prep education programs. Action-planning helps individuals or groups follow 
through on learning. Educational leaders should ask the question, “Based on what we have 
learned, what should we do now?” Action-planning can occur at a state or local consortium level 
and may involve participants from secondary schools and two-year colleges. 

Action-planning starts by engaging practitioners in reflecting carefully on the results of the 
evaluation and putting the results into context. This process engages practitioners in identifying 
strengths and weaknesses in both the Tech Prep education program and the evaluation process. 
Following this reflection, action-planning participants should prioritize the issues and set new 
goals, focusing on activities that have a high impact on student and program outcomes. Priorities 
should be clearly linked to educational and legislative standards. 

Once the priorities are set, the action plan can be developed. Generally, the plan takes the 
form of a written commitment to take certain actions. A typical plan can be designed to suit the 
parties involved, and it will often describe goals, objectives, strategies, potential barriers, and 
needed resources (human, technical, and funding). The plan should clearly state deliverables, 
responsibilities, and timelines for all activities and stakeholders. Criteria and monitoring methods 
(e.g., who, when, how) should be specified. Ultimately, the plan should provide the framework 
for achieving the goals. 

Planning of this type has occurred across the country by Tech Prep consortia. For 
example, Mid-Minnesota's School-to-career/Tech Prep consortium has been incorporating 
action-planning since 1994 (Schroeder, 2000). The planning starts at the local consortium with 
the Tech Prep leadership team identifying the goals for each of Minnesota's seven Tech Prep 
indicators. The Tech Prep leadership team identifies the strategies to assist the nine secondary 
schools within the consortium to develop their local plans. Each secondary school has the 
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flexibility of identifying the individuals responsible for completing the action plan. The plan is 
designed to meet the individual secondary school's needs. Resources are identified for each 
strategy. During the spring, summaries of the action-planning results are submitted to the local 
consortium Tech Prep director. The nine secondary schools meet to discuss their individual 
activities and the goals achieved. 

Continuous Quality Improvement 

To ensure that evaluation and action-planning link to future Tech Prep education program 
improvements, institutions can use a continuous quality improvement approach. Continuous 
quality improvement (CQI) is the continuous monitoring and modifying of programs to upgrade 
quality. Institutions should build in systems and structures that support and promote CQI within 
their programs and evaluation models. CQI stems from a range of sources including the quality 
movement, total quality management, and the Japanese kaizen view of quality. 

Kaizen focuses on the process rather than the results (Dixon & Swiler, 1990). It involves 
everyone in the organization to maintain and improve programs. Tech Prep can apply this 
philosophy by involving a range of consortium members in CQI and on-going program review. 
The action-planning team could also assist with these efforts. 

To understand CQI, the term quality needs to be defined. Numerous definitions of quality 
exist. Proctor and Gamble defines quality as “the unyielding and continually improving effort by 
everyone in an organization to understand, meet and exceed the expectations of customers” 
(Bragg, 1992, p. 16). In Japan, quality often means “anything that can be improved” (Dixon & 
Swiler, 1990, p.18). Tech Prep's definition of quality may be defined by legislation or local, 
regional, or state policies. Data collected to evaluate Tech Prep education programs should 
document state or local consortia quality indicators. 

The National Association of Tech Prep Leadership (NATPL) has developed a list of quality 
indicators to provide a consistent vision for Tech Prep education programs. The quality 
indicators are written for five integral Tech Prep program components: (a) accountability/ 
sustainability, (b) student opportunities, (c) curriculum, (d) articulation, and (e) professional 
development. These indicators can be used as a benchmark for Tech Prep continuous quality 
improvement. A copy of the quality indicators is provided in Appendix S. 

Texas has developed a minimum of two quality measures for each of the 10 review areas 
included in the Tech Prep consortium site visit (see Appendix G). A site visit includes the review 
of each measure. The reviewer rates each measure as “meets standard” or “does not meet 
standard.” As a result, measures meeting the standard can be expanded, and measures that do not 
meet the standards can be addressed and corrected. This process permits the consortium to be in 
a continuous quality improvement cycle. 
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Total Quality Management 

The total quality management (TQM) movement has been valuable in shaping CQI. TQM is 
a set of management practices that aims for total quality, that is, where the customers' 
requirements are incorporated into the product design or service (Brunetti, 1993). Brunetti's 
principles include (a) putting the customer first; (b) managing by fact; (c) planning, doing, 
checking, and acting, or using a systematic approach to improve; (d) focusing on prevention; (e) 
promoting employee involvement in the process; and (f) using cross-functional management, that 
is, cooperating across functional boundaries. For Tech Prep, these principles translate to building 
the customer (e.g., student, parent, stakeholder, teacher, business partner, consortium) into the 
design and delivery of Tech Prep education programs. One strategy for implementing these 
principles is to include all key stakeholders in the action-planning team. In TQM “a fundamental 
concept... is that outputs are never good enough; the threshold for quality is continually raised” 
(Bragg, 1992, p. 20). The principles of TQM can be used as a tool in planning continuous quality 
improvement into Tech Prep education programs. 

Florida's Tech Prep evaluation model incorporates the TQM model (Hammons, 1995). The 
model includes three operational components to encourage the ongoing process of continuous a 
quality improvement: (a) documentation, (b) analysis, and (c) improvement. Documentation 
includes the gathering of data related to the development and implementation of Tech Prep goals, 
completion of a Tech Prep self-study, and assessment of eight Tech Prep essential program 
elements. The analysis component involves analyzing the data collected from the documentation 
process. Quantifiable goals and benchmarks are reported for each consortium. The third 
component, improvement, involves implementing change as part of future program planning. 
Consortia recommend areas for improvement and implement a plan for change. The 
improvement phase is monitored for overall impact. 

Malcolm Baldrige Strategies 

The Malcolm Baldrige National Quality Award twww.baldrige.org) provides criteria as a 
management guide for quality improvement in America. Ross (1993) states that the common 
themes of the Baldrige award are customer-driven quality, continuous improvement, 
measurement, participation, leadership, and management by data (rather than experience or 
intuition). 

The Baldrige Education Criteria for Performance Excellence provide a valuable framework 
to assess and measure. Seven criteria are used to assess education: (a) leadership, (b) strategic 
planning, (c) student and stakeholder focus, (d) information and analysis, (e) faculty and staff 
focus, (f) educational and support process management, and (g) organizational and performance 
results. The criteria focus on five organizational performance areas: (a) student performance 
results, (b) student and stakeholder focused results, (c) budgetary and financial results, (d) 
faculty and staff results, and (e) organizational effectiveness results. The criteria and 
performance areas can be integrated into Tech Prep to enhance CQI. 
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In Minnesota, a Tech Prep Self-Evaluation System model (Pucel, Brown, & Kuchinke, 1996) 
was designed using the Malcolm Baldrige National Quality Award criteria. The model “was 
designed to gather data on program outcome measures to monitor actual consortium 
productivity, and to gather self-evaluation data as a basis for program improvement” (p. 82). Key 
stakeholders were involved in the process, and data were collected to help consortia effectively 
identify areas of improvement. Tech Prep evaluators can use these guidelines to assist in 
developing CQI strategies. 



Communicating Evaluation Results 

As with any new initiative, those promoting change must be careful to educate and include 
others in the efforts. Planners must be sure to gain the commitment and involvement of 
stakeholders. Including stakeholders in the evaluation process facilitates this. Communication 
should occur frequently among stakeholders to allow for questions to be asked and information 
to be distributed. Other means of communication may include a Web site, a quarterly stakeholder 
meeting, a monthly Tech Prep newsletter, or a Tech Prep annual report. Informal methods of 
communication can also be promoted, such as telephone conversations and e-mail. The Action- 
Planning Team (APT) should consider creating a specific communications or marketing role 
within the committee or consortium. Finally, the APT must monitor the implementation of the 
action plan. Specific responsibility for monitoring should be assigned to an individual person 
who will be required to review the action plan and measure the progress against goals on an 
annual basis. 



Summary 

After an initial evaluation is completed, it is important to review the results, identify future 
program plans, and set new goals for program improvement. Tech Prep consortia should build 
systems and structures to support and promote continuous improvement within the Tech Prep 
education program. In Section V, four strategies were presented to link evaluation with program 
improvement. The principles of continuous improvement should be viewed as a tool to assist 
with program improvement. Evaluation results should be communicated to decision makers 
when they will most likely need them. Communicating results is one way to educate and inform 
others about Tech Prep efforts at the state and local level. 
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GLOSSARY 

Baseline - current level of performance, a specific value that can serve as a basis for comparison 
or judgment. 

Benchmark -the optimal level of performance for comparison or judgment. 

Career clusters - 16 clusters have been created by the U.S. Department of Education, Office of 
Vocational and Adult Education, to assist students in planning their pathways to college and/or 
employment. Examples of career clusters include information technology, business and 
administration, and arts, audio video technology and communications. 

Career guidance and academic counseling - access to information regarding career awareness 
and planning with respect to an individual's occupational and academic future that involves 
guidance and counseling with respect to career options, financial aid, and postsecondary options 
(Carl D. Perkins Vocational and Applied Technology Act, 1998). 

Career pathways - career and technical education programs that link related courses of study 1 
from secondary school through two-year colleges. College-level credit can be earned by 
completing courses at the secondary level. Career pathways are unique to program and .course 
offerings in a local consortium. 

Career and technical education - prepares both youth and adults for a wide range of careers, 
from registered nurse to computer technician, that require varying levels of education-from high 
school to postsecondary certificates to two-and four-year college degrees (Retrieved March 17, 
2001. from the World Wide Web: http://www.acteonline.org/faqs/faqs.htmn . 

Cohort- a group of individuals having a statistical factor (e.g., age or class membership) in 
common in a demographic study. 

Cooperative education - a method of instruction of education for individuals who, through 
written cooperative arrangements between a school and employers, receive instruction, including 
required academic courses and related vocational and technical education, by alternation of study 
in school with a job in any occupational field; the alternation shall be planned and supervised by 
the school and employer so that each contributes to the education and employability of the 
individual, and may include an arrangement in which work periods and school attendance may 
be on alternate half days, full days, weeks, or other periods of time in fulfilling the cooperative 
program (American Vocational Association, 1998, p. 88). 

Core indicator framework - the framework developed by the Office of Vocational and Adult 
Education to assist states with performance accountability for vocational and technical education. 
The framework addresses the 4 Perkins core indicators and 15 subindicators. It defines the goals, 
performance measures, and state measurement approaches for each of the 1 5 subindicators. 
Additionally it defines the quality criteria (validity, reliability, cost-effectiveness, system-focused 
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and management utility) for assessing state measurement approaches (Office of Vocational and 
Adult Education, 2000a). 

Displaced homemaker - an individual who (a) has worked primarily without remuneration to 
care for a home and family, and for that reason has diminished marketable skills; has been 
dependent on the income of another family member but is no longer supported by that income; or 
is a parent whose youngest dependent child will become ineligible to receive assistance under 
part A of title IV of the Social Security Act (42 U.S.C. 601 et seq.) not later than 2 years after the 
date on which the parent applies for assistance under this title; and (b) is unemployed or 
underemployed and is experiencing difficulty in obtaining or upgrading employment (American 
Vocational Association, 1998, p. 88). 

Evaluator - individuals responsible for planning and conducting an evaluation in order to judge 
the value or effectiveness of a program. 

Individual with limited English proficiency (LEP) - a secondary school student, an adult, or an 
out-of-school youth who has limited ability in speaking, reading, writing, or understanding the 
English language, and (a) whose native language is a language other than English; or (b) who 
lives in a family or community environment in which a language other than English is the 
dominant language (American Vocational Association, 1998, p. 89). 

Individual with a disability - an individual with any disability (as defined in section 3 of the 
Americans with Disabilities Act of 1990 (42 U.S.C. 12102) (American Vocational Association, 
1998, p. 89). 

Internship - program organized for students or recent graduates at the secondary or 
postsecondary levels to undergo supervised practical training in the workplace. 

Management utility - the degree to which measurement approaches are useful in managing 
continuous improvement at the state and local levels (Office of Vocational and Adult Education, 
2000a, p. 9). 

Nontraditional training and employment - occupations or fields of work, including careers in 
computer science, technology, and other emerging high-skill occupations, for which individuals 
from one gender comprise less than 25 percent of the individuals employed in each such 
occupation or field of work (American Vocational Association, 1998, p. 90). 

Performance measures - the definition of the performance measures for each subindicator 
including the definition of the numerator and denominator of each performance measure (Office 
of Vocational and Adult Education, 2000a, p. 7). 

Performance measurement approaches - a state approach for each performance measure. These 
approaches include assessment and data collection strategies (Office of Vocational and Adult 
Education, 2000a, p. 8). 
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Performance standards - the minimum standards of performance required of a Tech Prep 
student or program. Performance standards (e.g., 80% retention rate) may be set by the state, 
consortium, or institution for a specific time frame (e.g., school year). Performance standards can 
be developed based on baseline data, benchmarks, criteria, or previous performance. 

Postsecondary educational institution - (a) an institution of higher education that provides not 
less than a two-year program of instruction that is acceptable for credit toward a bachelor's 
degree; (b) a tribally controlled college or university; or (c) a nonprofit educational institution 
offering certificate or apprenticeship programs at the postsecondary level (American Vocational 
Association, 1998, p. 90). 

Practitioner - administrators, teachers, counselors, trainers, workplace supervisors, and other 
professionals working with Tech Prep education programs. 

Program outcomes - benefits or changes for individuals or populations during or after 
participating in program activities. Outcomes may relate to behavior, skills, knowledge, attitudes, 
values, condition, or other attributes. Program outcomes may include Tech Prep student 
knowledge and skill levels, employment rates for Tech Prep students, or employer satisfaction 
with Tech Prep graduates. 

Quality criteria for performance measurement - the quality criteria for performance 
measurement to ensure sufficient rigor and comparability of state performance measurement and 
reporting (Office of Vocational and Adult Education, 2000a, p. 8). 

Reliability- the degree to which performance measurement is conducted in a consistent manner 
using standardized or consistent data collection instruments (e.g., student record forms, surveys, 
assessment instruments) and effective management information systems for insuring data quality 
(Office of Vocational and Adult Education, 2000a, p. 8). 

School dropout - an individual who is no longer attending any school and who has not received 
a secondary school diploma or its recognized equivalent (American Vocational Association, 

1998, p. 90). 

Special populations - (a) individuals with disabilities; (b) individuals from economically 
disadvantaged families, including foster children; (c) individuals preparing for nontraditional 
training and employment; (d) single parents, including single pregnant women; (e) displaced 
homemakers; and (f) individuals with other barriers to educational achievement, including 
individuals with limited English proficiency (American Vocational Association, 1998, p. 90). 

Stakeholder - individuals and groups who have a direct interest in and may be affected by the 
program being evaluated or the evaluation's results (Worthen, Sanders, & Fitzpatrick, 1997). 

Student outcomes - changes that occur in individuals as a result of participation in an 
educational experience. Frequently expressed in terms of the value that is added to an individual 
as a consequence of that participation (Bragg, 1992). 
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Subindicator goals - the long-term vision or goal statement for each subindicator -that is, what 
we hope to achieve in the future through continuous improvement of vocational education 
(Office of Vocational and Adult Education, 2000a, p. 7). 

Tech Prep essential program elements — required for Tech Prep education programs: 

(a) articulation agreement, (b) appropriate curriculum design, (c) curriculum development, 

(d) in-service teacher training, (e) counselor training, (f) equal access for special populations, and 
(g) preparatory services (American Vocational Association, 1998). 

Threshold level of vocational education — a program or sequence of courses or instructional 
units that provides an individual with the academic and technical knowledge, skills, and 
proficiencies to prepare the individual for employment and/or further or advanced education 
(Office of Vocational and Adult Education, 2000a, p. 10). 

Validity - the degree to which a performance measurement approach directly and fully measures 
program and student outcomes at an appropriate interval. Produce valid data when they use 
assessment and other data collection instruments that have strong content validity (Office of 
Vocational and Adult Education, 2000a, p. 8). 

Vocational and technical education - organized educational activities that (a) offer a sequence 
of courses that provides the academic and technical knowledge and skills that individuals need to 
prepare for further education and for careers (other than careers requiring a baccalaureate, 
master's, or doctoral degree) in current or emerging employment sectors; and (b) include 
competency-based applied learning that contributes to the academic knowledge, higher-order 
reasoning, and problem-solving skills, work attitudes, general employability skills, technical 
skills, and occupation-specific skills of an individual (American Vocational Association, 1998, p. 
92). 
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APPENDIX A 

Template for Perkins Core Indicator 



CORE INDICATOR 1 - Student Attainment 



Sub- 


Goal 


Measure 


Measurement Approach 


Quality Criteria for 


indicator 








Measurement Approach 


1S1. 


All students 


Numerator: 


State Academic Assessment 


Alignment to State 


Secondary 


who reach a 


Number of 


Svstem- State defined 


Academic Standards- 


Academic 


state-defined 


students 


performance benchmarks on 


Attainment measures and 


Attainment 


threshold 


reaching a state- 


state-developed academic 


assessment systems are 




level of 


defined 


assessment systems used in state 


aligned to state academic 




vocational 


threshold level 


educational accountability 


content and performance 




education 


of vocational 


systems including high school 


standards. 




will master 


education and 


graduation qualification 


Scope of Attainment 




academic 


who have met 


examinations. 


Measurement- Attainment 




knowledge 


state academic 


National Standardized Academic 


measures address all of 




and skills that 


standards and 


Assessments - State defined 


the core academic content 




meet 


have left 


performance benchmarks on 


areas (language arts, 




challenging 


secondary 


national standardized 


mathematics, social 




state 


education in the 


assessments. 


studies, science) 




academic 


reporting year. 


Academic Course Completion- 


addressed in state 




standards. 




State-defined performance 


academic standards and 








benchmark (e.g. grade, certificate) 


assessed in state 






Denominator: 


in designated academic courses 


assessment systems. 






Number of 


including integrated 


Timing of Attainment 






students 


academic/vocational courses. 


Measurement- Attainment 






reaching a state- 


Academic Grade Point Average- 


is measured concurrent 






defined 


State -defined grade point average 


with or after concentrated 






threshold level 


for designated academic courses 


participation in vocational 






of vocational 


including integrated 


education. 






education and 


academic/vocational courses. 


Reliability of Attainment 






who have left 


Embedded Academic Assessment 


Measurement- Attainment 






secondary 


in Vocational/Integrated 


is measured using reliable 






education in the 


Courses/Programs - Performance 


assessment instruments 






reporting year. 


benchmark on course or program 


that are administered 








assessments in integrated courses 


consistently in assessment 








and programs. 


systems. 








Overall Grade Point Average- 


Student Coverage in 








State defined grade point average 


Attainment Measurement- 








for all courses in the school or 


Performance 








program. 


measurement reports 








High School Graduation/Program 


attainment data for all 








Completion- Graduating from 


students reaching state- 








high school or completing a 


defined threshold levels in 








program (when graduation or 
completion is the same as 
attaining state- or program - 
defined academic standards. 


the state. 



Permission to reprint material obtained from V. Schray, U.S. Department of Education, Washington DC. 
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APPENDIX B 

Nebraska's Suggested Tech Prep Performance Measures and Possible Standards 



Type of Outcome 


Measures 


Possible Standards 


Learning Outcome 


High school Tech Prep education 
program completion. 


More than 45% of all high school 
graduates complete a Tech Prep 
program sequence of courses. 




Community college degree, two- 
year certificate completion. 


Completion rate greater than 60% 
for all who are admitted within 5 
years. 




Combined sequence of academic 
and vocational courses. 


All high school graduates will have 
2 years in a sequential vocational 
education program with at least 2 
years of related, applied academics. 




Basic and advanced skills 
competency. 


90% of community college enrollees 
will meet the ASSET admissions 
standards. 






100% of Tech Prep education 
program completers will 
demonstrate basic and advanced 
academic skill gains commensurate 
with program objectives, or the 
student's individualized education 
plan as detennined at the local level. 






Tech Prep graduates will 
demonstrate higher gains in basic 
and academic skills compared to 
nonprogram students. 




Occupational skills competency. 


90% of Tech Prep students will 
attain 1 00% of the occupational 
skills identified for that program at 
the local level. 




Advanced placement in community 
college. 


30% of beginning Tech Prep 
students will gain advanced 
placement or receive a comparable 
amount of postsecondary credit. 




Equity in learning outcomes for 
special population students. 


Learning outcome rate for special 
population students are comparable 
to the rates for the total population. 




Gender equity outcomes. 


Learning outcome rates for males 
and females are comparable. 
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Type of Outcome 


Measures 


Possible Standards 


Placement Outcomes 


Community college enrollment 
following high school. 


Enrollment rate greater than 35% for 
all Tech Prep graduates within one 
year of graduation (2 T 2). 




Higher education enrollment. 


Enrollment rate of community 
college graduates greater than 30% 
within 5 years of graduation (2 + 2 + 
2). 




Employment in related occupation. 


More than 50% of Tech Prep 
graduates obtain employment in a 
related occupation within 1 year of 
graduation. 




Transition to Apprenticeship. 


30% of Tech Prep students will have 
entered into an apprenticeship 
certified by the Bureau of 
Apprenticeship and Training. 




Equity in placement outcomes for 
special population students. 


Post-program placement rates for 
special population students are 
comparable to the placement rates 
for the total population. 




Gender equity outcomes. 


Post-program placement rates for 
males and females are comparable. 



0 
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APPENDIX C 

Examples of Tech Prep Student, Concentrator, and Completer Definitions 



Tech Prep Student 


Secondary Definitions 


Postsecondary Definitions 


AZ 


A Tech Prep student at the secondary 
level is one who is enrolled in a program 
during high school which encompasses 
an approved vocational program of 
study articulated with a community 
college for preparation in a technical 
career, as well as an academic core 
program which includes a minimum of 
four years of English, two years of 
mathematics and two lab science 
courses. Core academic courses may be 
existing college prep courses and/or 
courses specifically designed to provide 
high level academic skills within a 
vocational context. While not required, 
the student may be participating in a 
school-to-work transition program or 
registered apprenticeship program as 
well. 


A Tech Prep student at the postsecondary 
level is one who is enrolled in an associate 
degree or certificate program that articulates 
with a secondary vocational program 
completed by the student or for which 
specific vocational courses taken at the 
secondary level result in adequate 
preparation for this degree or certificate. In 
addition, a Tech Prep student may be one 
whose associate degree or certificate 
program includes continuation of an 
articulated school-to-work program, and/or 
one who is in a registered apprenticeship or 
certificate program. 


FL 


A student enrolled in an articulated, 
sequential program of study (enrolled in 
level 2 or above courses) at grade level 
or above by grade 1 1 in mathematics, 
science, and communications, including 
a technical component, which leads to a 
minimum of a two-year postsecondary 
certificate or degree and/or 
apprenticeship program. 




IL 


One who has made a conscious decision 
to follow a clearly defined sequence of 
courses to prepare for employment in a 
Tech Prep occupation. The student has 
declared Tech Prep as a major and has 
an Individualized Career Plan indicating 
a Tech Prep occupation as a career goal. 
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MA 


A student who has indicated an intent to 
pursue, and is enrolled in courses within 
a recognized Tech Prep education plan 
that consists at a minimum of 2 years of 
secondary and 2 years of postsecondary 
study; is carried out under a written 
articulation agreement; allows the 
student to earn postsecondary credit 
while in secondary school; and leads to 
a specific postsecondary 2-year 
certificate, degree or apprenticeship. 


A student who has participated in the 
secondary portion of a recognized Tech 
Prep education plan and has enrolled in the 
postsecondary portion of that education plan 
and may have transferred in college credit 
earned in secondary school. 


MI 


A student who has an Educational 
Development Plan that indicates that 
he/she is enrolled in a specific Tech 
Prep (2+2) Program curriculum leading 
to an Apprenticeship, Certificate or 
Associate degree in a specific career 
field. 


A student who has an Educational 
Development Plan that indicates that he/she 
is enrolled in a specific Tech Prep (2+2) 
Program curriculum leading to an 
Apprenticeship, Certificate or Associate 
degree in a specific career field. 


MN 


A secondary student who is enrolled in a 
Tech Prep technical course(s) of 91-180 
hours within a recognized Tech Prep 
program. 




MO 


A student engaged in the Tech Prep 
Education Program is formally counted 
as a Tech Prep student when they 
initially enroll in the Department of 
Elementary and Secondary Education’s 
approved vocational course(s) portion of 
the program and when they make a 
commitment to pursuing completion of 
an associate or baccalaureate degree, 
post-secondary certificate, or two more 
year apprenticeship program in a 
specific field of study. 




NC 


All students are targeted. As explained 
for career pathways, all students “need 
both the strong academic foundation and 
the technical skills to be successful in 
today’s world.” So, all students should 
design their course of study based on the 
recommended courses for a career path 
that would lead to a chosen career. 




NE 


A student who is enrolled in one or 
more applied or technical courses that 
are delineated in a Tech Prep intent to 
articulate or articulation agreement. 


A student who is enrolled at the community 
college and has completed one or more 
applied or technical secondary school 
courses that are delineated in a Tech Prep 
intent to articulate or articulation agreement. | 
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NY 


Any junior or senior who was selected 
as part of a recruitment process and 
participated in a minimum of Tech Prep 
math and Tech Prep English courses, a 
technical career cluster and transition to 
technical college activities. 




OH 


One who is enrolled in a sanctioned 
Tech Prep program, beginning in grade 
1 1 and continuing through the associate 
degree in the occupational and 
employability competency delivery 
system. 




OK 


l)The student has chosen a career 
cluster from those that have been 
identified, developed, and implemented 
by the tech prep consortium. 2)The 
individual student plan of study has 
been developed with the assistance of a 
counselor, advisor, or another person 
familiar with the tech prep mission. The 
parent or guardian must be given the 
opportunity to participate in plan 
development. The individual plan of 
study must consist of a non-duplicated 
sequence of courses linking secondary 
education to postsecondary education 
and may include a worksite experience 
component including a youth or 
registered apprenticeship. 3)The career 
cluster chosen by the student requires 
technical preparation. 4)The plan of 
study for the career cluster chosen by 
the student results in a postsecondary 
outcome such as licensure, certification, 
or an associate/baccalaureate degree. 


-V 


OR 


A student who chooses, in his/her junior 
or senior year, to enroll in a major 
course of study in a 2+2 Tech Prep 
program. Tech Prep students must earn 
a minimum of two credits in 
professional-technical courses between 
their junior and senior year, and these 
programs should be linked to a two-year 
associate degree program at the 
community college. 
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TN 


A student who has a four or six year 
plan that includes participating in an 
approved official Tech Prep program 
designed specifically to prepare the 
student for a postsecondary degree, 
certificate, or approved apprenticeship 
leading to employment in related 
technical career fields. To be approved, 
a Tech Prep program must be governed 
by a consortium between secondary and 
postsecondary educational institutions 
with a formal articulation agreement. 




TX 


A student in grades 9-12 who follows an 
approved Tech Prep high school plan of 
study leading to postsecondary 
education and training, and is enrolled 
in courses appropriate to that plan. 


A student who declares a major leading to 
an associate of applied science degree that 
is state approved as Tech Prep. 


WI 


A Tech Prep student is defined as any 
student who chooses to access career 
counseling, career exploration or 
technical preparation course of study. A 
technical preparation course of study is 
sequential and combines at least two 
years of secondary and two years of 
postsecondary education leading to an 
associate, or a baccalaureate degree, a 
postsecondary certificate in a specific 
career field, or placement in appropriate 
employment or further education. 




WY 


A student who has taken one semester 
course and is currently taking a second 
semester course in a vocational program 
or has taken two or more semester 
courses in a vocational program. 


A student who has taken two semester 
courses and is currently taking a third 
semester course in a vocational program or 
has taken three or more semester courses in 
a vocational program. 
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Tech Prep Concentrator 


Secondary Definitions 


Postsecondary Definitions 


CT 


Students who have taken a sequence of 
two or more credits in a single 
vocational and technical area (a credit 
equals one full year). 




ID 


A student who has completed three or 
more semesters of a professional- 
technical program sequence by the fall 
of his/her junior year. 




LA 


A student who has attained the academic 
and technical knowledge / skills / 
proficiency with a program/sequence of 
courses or instructional units that 
provides an individual with the 
academic and technical knowledge / 
skills / proficiencies to prepare the 
individual for employment and/or 
further/advanced education. 




MS 


Students who have crossed the threshold 
of participation by completing three or 
more courses or equivalent hours of 
instruction in a single career or industry. 


Students who are enrolled in vocational 
and technical education certificate or 
degree programs. 


MT 


A student who has received at least 
three units of vocational course credit 
during a high school career. A unit of 
credit equals two semesters of study. 


A student who declares a program of study 
in vocational / technical education. 


NM 


Those students enrolled in an identified 
program or coherent sequence of 
courses or instructional units providing 
them with the academic an technical 
knowledge, skills and proficiencies to 
prepare for employment and/or 
further/advanced education and students 
who have completed three or more 
career-technical courses in that sequence 
(secondary). 


Those students enrolled in an identified 
program or coherent sequence of courses 
or instructional units providing them with 
the academic an technical knowledge, 
skills and proficiencies to prepare for 
employment and/or further/advanced 
education and students who have 
completed nine postsecondary credit hours, 
and have declared a career-technical 
education major (postsecondary). 


OR 


A student who has accumulated at least 
two credits in an approved professional 
technical education program during the 
four years of high school. 


A student who has completed more than 
half of a state approved professional 
technical education certificate or degree 
program. 
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RI 


A 9'\ 10 th , 1 1 th or 12 th grade student who 
has indicated an intent to pursue and has 
enrolled in, and has completed at least 
two semesters of courses in either a)a 
state recognized career and technical 
education program... or b) a sequence of 
recognized courses in an educational 
plan that consists at a minimum of 2 
years of secondary study and 2 years of 
postsecondary study which is carried out 
under a written articulation agreement 
which allows the student to earn 
postsecondary credit while still in 
secondary school, and leads to a specific 
postsecondary 2 year certificate degree 
or apprenticeship, or high-skilled 
employment. 




WY 


A student who has taken three or more 
semester courses in a vocational 
program, including those who may be 
currently enrolled in their third course. 


A student who has taken six or more 
semester courses in a vocational program, 
including those who may be currently 
enrolled in their sixth course. 
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Tech Prep Completer 


Secondary Definitions 


Postsecondary Definitions 


CA 


A student who has completed a capstone 
vocational course- typically the third in a 
sequence of courses offered in a 
vocational program area. 




IL 


A student who follows a written career 
plan indicating a Tech Prep occupation as 
a career goal, and takes a sequence of 
program specific core academic and 
technical courses prepare for a Tech Prep 
occupation during two years of 
secondary school preceding graduation. 


A student who articulated to a non- 
duplicative academic and technical course 
of study for a Tech Prep occupation leading 
to an associate of applied science degree, 
two-year certificate, or two-year 
apprenticeship following high school, and 
completes two years of postsecondary 
education which led to an associate of 
applied science degree, two-year certificate, 
two-years of an apprenticeship, or 
articulated from a community college to a 'i 
four-year baccalaureate degree in a Tech 
Prep occupation. 


MA 


A student who has participated in both 
the secondary and postsecondary portions 
of the recognized Tech Prep education 
plan and has received an appropriate 
postsecondary 2-year degree, certificate 
or apprenticeship license. 


A student who has participated in a different 
associate degree program in college than 
his/her secondary Tech Prep program and 
receives a postsecondary degree, 2-year 
certificate or license. 


NJ 


A student who has attained the vocational 
and technical skills within a state- 
approved occupational program that 
provides the individual with the 
academic and technical knowledge/skills/ 
proficiencies to prepare the individual for 
employment and/or further/advanced 
education. 




OH 


A student who has followed an approved 
Tech Prep curriculum pathway and has 
earned an associate degree with an 
advanced skills certificate. 


A student who has enrolled in and 
completed an approved workforce 
development program and has demonstrated 
sufficient mastery of their vocational and 
academic subject matter to prepare them for 
their career and life-long learning goals as 
set forth in their individual career plan, and 
is no longer enrolled in secondary school. 
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RI 


Students who have completed course 
requirements for either of the following: 
a) a state recognized career and technical 
education program, or b) a sequence of 
recognized courses in an education plan 
consisting at a minimum of 2 years of 
secondary study carried out under a 
written articulation agreement allowing 
the student to earn postsecondary credit 
while still in high school, and leading to 
a specific postsecondary 2 year 
certificate degree, apprenticeship, or high 
skilled employment. 
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APPENDIX D 

Generic Tech Prep Articulation Agreement (OR) 



between 

Mt. Hood Community College 

and 

Participating High School 



WHEREAS, Mt. Hood Community College (MHCC) is the program community college serving students at 
High School; and 

WHEREAS, MHCC has an established Division that offers a 

program to students; and 

WHEREAS, the resources of these two education agencies may be shared for the mutual benefit of both programs 
and the benefit of those students served; and 

WHEREAS, each education agency has determined that the students can be served through a tech prep articulation 
program in for the purpose of providing students with linked educational opportunities. 

THEREFORE, it is hereby resolved that the MHCC Division and High 

School mutually agree to the conditions set forth in this agreement to grant tech prep college credit for 
High School classes listed on the agreement addendum approved by MHCC. 

MHCC tech prep credits are available to High School students as follows: 

• credit hours of tech prep credit will be awarded upon successful completion of and approval by MHCC 

• tech prep credit awarded as part of this agreement will be applicable to the degree program listed in the current MHCC 

catalog. Participation in tech prep does not automatically enroll a person to the named college program. 

• tech prep credit will be transcripted by MHCC for an annual registration fee of $25. All tech prep and Early Collegiate Opportunity classes 
that a student registers for in the course of the school year are covered by this registration fee. Regular tuition for credit granted under this 
agreement will be waived. 

Services provided by Mt. Hood Community College 

1 . provide a staff member to function as the contact person and coordinator for the high school. The coordinator will be 

responsible for implementing the provisions of this agreement. 

2. program staff will assist the High School with continuation of the program. 

Services provided by High School: 

1 . a instructor will be designated and responsible for implementing the provisions of this agreement. 

2. maintain a qualified program as defined by Oregon Department of Education program standards. 

3. inform and advise students about tech prep credit available through successful completion of the curriculum. 

4. submit necessary student tech prep registration materials and transcripting fees as required by MHCC Admissions and Records office to 
record earned tech prep credit. 

Both parties are obligated to meet to review this agreement annually. The agreement shall remain in effect until 
there is mutual agreement to terminate or until terminated by either party with written notice of cancellation. Such 
termination will become effective at the end of the term in which the cancellation occurs. 



Permission to reprint material obtained from the Tech Prep Consortium Director, Mt. Hood Community College, 
OR. 
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Tech Prep Articulation Agreement Addendum (OR) 

Between 

Mt. Hood Community College 

And 

High School 



The following 

Hood Community College course, as taught by the listed instructor: 



Course Name 



High School course(s) have been approved as equivalent to the listed Mt. 



MHCC 

Course Course Name 
Number 



High School 



Course Name Instructor Course Length 



MHCC credit will be awarded to high school students upon successful completion of MHCC 
approved high school course(s) listed above with at least a “B” final grade and recommendation 
of the high school instructor. 



0 

ERJC 



High School Principal or Designee 


Date 


MHCC Divisional Representative 


Date 


MHREC Regional Coordinator 


Date 


Permission to reprint material obtained from the Tech Prep Consortium Director, Mt. Hood Community College, 
OR. 
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Secondary Membership Administrative Agreement (MT) 

Tech Prep Consortium 

INTENT 

This agreement establishes membership in the Tech Prep Consortium and affirms the clear 

intention of implementing and promoting Tech Prep efforts in our District. Participating educational entities include 

as the lead postsecondary institution and the high school represented on this 

agreement. a member of the consortium, will be responsible for the fiscal 

management of the Tech Prep program. This cooperative endeavor will provide focused and respected learning opportunities that link secondary 
and postsecondary curricula in order to eliminate duplication and provide students a progressive path of education and training leading to entry 
and success in the rapidly expanding range of technical occupations which require postsecondary technical education and preparation up to the 
Associate Degree or Certificate level. 

The program concept and philosophy envisioned for Tech Prep in Montana is described in the Definition which follows. Continuing 
development and implementation of the Tech Prep initiative will be accomplished through objectives affirmed by the Consortium's Tech Prep 
Leadership Team or Advisory Panel. 

DEFINITION 

Montana Tech Prep coordinates secondary and postsecondary programs to ensure all students the opportunity to effectively prepare for 
ongoing learning and the changing workplace. It facilitates smooth transitions for students pursuing postsecondary options leading to the 
completion of an associate degree, certificate, or apprentice program while meeting high academic and technical standards. Montana Tech Prep 
systems currently focus on: 



• Articulation Agreements 

• Career planning and guidance 

• Professional development of faculty and staff 

• Program and curriculum development 

• Cooperation with community, business, and industry 

• Technical preparation in a minimum of one career field 

• Outreach and public awareness 



MISSION STATEMENT 

The mission of the Technical Preparation (Tech Prep) Program is to assure that all students acquire the academic and technical skills and 
knowledge necessary to be prepared for life-long learning and employment in the technology oriented society of the future. 

STATEMENT OF AGREEMENT AND COMMITMENT TO REPORT 

The Board of Trustees of recognizes the value of 

(School District) 

two-year alternatives and postsecondary technical education as well as the need to help students make a smooth transition from the secondary to 
the postsecondary level without unnecessary delay or duplication. 

We support the pursuit, maintenance, and refinement of articulation agreements that will reduce duplication between secondary and 
postsecondary courses and will promote students' progressive achievement of specific competencies in programs that lead to employment m 
vocational and technical fields, including business, health, and mechanical/industrial/practical arts or trades. We understand that the agreements 
reached through this effort relate to programs of study leadmg to associate degrees or certificates and that students meetmg conditions of these 
agreements are eligible for Tech Prep credit at participating colleges. 

The Board of Trustees declares its desire and intent to participate as an active member of this Tech Prep Consortium. We encourage the 
administration and faculty at the District’s high school(s) to work with the Consortium to establish and support articulated Tech programs and 
related student services. 

In compliance with reporting requirements that substantiate meaningful use of Perkins federal funds, document benefits extended to special 
populations, and assist in the preparation of Montana's annual report to the U.S. Department of Education, high school and District staff will 
provide requested data about the numbers and characteristics of students being served by Tech Prep programs and services. 



Chairman of the Board of Trustees or Superintendent of Schools District 



Date 



Principal High School Date 

Permission to reprint material obtained from the Montana Tech Prep Coordinator, MT. 
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Postsecondary Membership Administrative Agreement (MT) 



INTENT 



Tech Prep Consortium 



This agreement establishes membership in the Tech Prep Consortium and affirms the clear intention of implementing 

and promoting Tech Prep efforts in our District. Participating educational entities include as the lead 

postsecondary institution and the high school represented on this agreement. , a member of the consortium, will 

be responsible for the fiscal management of the Tech Prep program. This cooperative endeavor will provide focused and respected learning 
opportunities that link secondary and postsecondary curricula in order to eliminate duplication and provide students a progressive path of 
education and training leading to entry and success in the rapidly expanding range of technical occupations which require postsecondary technical 
education and preparation up to the Associate Degree or Certificate level. 

The program concept and philosophy envisioned for Tech Prep in Montana is described in the Definition which follows. Continuing 
development and implementation of the Tech Prep initiative will be accomplished through objectives affirmed by the Consortium's Tech Prep 
Leadership Team or Advisory Panel. 

DEFINITION 

Montana Tech Prep coordinates secondary and postsecondary programs to ensure all students the opportunity to effectively prepare for ongoing 
learning and the changing workplace. It facilitates smooth transitions for students pursuing postsecondary options leading to the completion of an 
associate degree, certificate, or apprentice program while meeting high academic and technical standards. Montana Tech Prep systems currently 
focus on: 



• Articulation agreements 

• Career planning and guidance 

• Professional development of faculty and staff 

• Program and curriculum development 

• Cooperation with community, business, and industry 

• Technical preparation in a minimum of one career field 

• Outreach and public awareness 



MISSION STATEMENT 



The mission of the Technical Preparation (Tech Prep) Program is to assure that all students acquire 
the academic and technical skills and knowledge necessary to be prepared for life-long 
learning and employment in the technology oriented society of the future. 



STATEMENT OF AGREEMENT AND COMMITMENT TO REPORT 

recognizes the value of two-year alternatives and postsecondary technical 

(College or Post -secondary Institution) 

education as well as the need to help students make a smooth transition from the secondary to the postsecondary level without unnecessary delay 
or duplication. 

This postsecondary institution supports the pursuit, maintenance, and refinement of articulation agreements that will reduce duplication between 
secondary and postsecondary courses and will promote students’ progressive achievement of specific competencies in programs that lead to 
employment in vocational and technical fields, including business, health, and mechanical/industrial/practical arts or trades. It is understood that 
the agreements reached through this effort relate to programs of study leading to associate degrees or certificates and that students meeting 
conditions of these agreements are eligible for Tech Prep credit at the participating colleges. 

I affirm the desire and intent of this college to participate as an active member of this Tech Prep Consortium. I encourage the college’s 
administration and faculty to work with the Consortium to establish and support articulated Tech Prep programs and related student services. 

In compliance with reporting requirements that substantiate meaningful use of Perkins federal funds, document benefits extended to special 
populations, and assist in the preparation of Montana's annual report to the U.S. Department of Education, staff from this college or institution 
will provide requested data about the numbers and characteristics of students being served by Tech Prep programs and services. 



Chief Executive Officer 



College or Institution 



Date 



Permission to reprint material obtained from the Montana Tech Prep Coordinator, MT. 
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APPENDIX E 

Tech Prep Consortia Annual Report (FL) 

I. General Information 

Directions: Complete the following information about the consortium. 

Consortium Fiscal Agent: 

Community College: 

School District(s): 

Schools: 



Report Contact: E-mail: 

Address: 

Title: Phone: Fax: 

II. Status of Required Tech Prep Components 

Directions: Please provide a brief response to the following three questions for each of the 
following Tech Prep Components: (1) Administration and Organization; (2) Articulation 
Agreements; (3) Business, Industry and Community Involvement; (4) Consortium Level 
Evaluation; (5) Curriculum Development/Programs of Study; (6) Promotions/Marketing; 
and (7) Staff Development. Additional information on another component. Impact on 
Students, is requested in Section III. 

1 . What have been your significant activities/successes in each component area? What 
strategies or approaches have proven to be beneficial? 

2. What barriers have you encountered in each component and how have vou addressed 
them? 

3. What are your current technical assistance needs as related to each Component? 

m. Impact on Students 

Directions: The following items 1-16 address your consortium's impact on students. 

Please answer the following questions and complete the data tables. 

1 . The state of Florida's definition of a Tech Prep Student is: 
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A student enrolled in an articulated, sequential program of study, * at grade level or 
above by grade eleven in mathematics, science, and communication, including a 
technical component, which leads to a minimum of a two-year post-secondary 
certificate or degree, and/or apprenticeship program. 

* Enrolled in level 2 or above courses. 

Does your consortium use this definition? 

Yes No : Our definition is : 

2. Did you implement the Tech Prep Participant data element in your school district's 
Management Information System (MIS) for Tech Prep students for the 1998-99 
academic year? 

Yes No Please explain: 

3. The table below requests a breakdown of Tech Prep students by grade level, gender, 
ethnicity, and special population. Please provide the demographic information to 
complete the following table: 



DEMOGRA 


PH1C CHAR 
Grade 9 
No, % of 
total 


ACTERISTICS 
Grade 10 
No. % of 

total 


; OF TECH PREI 
Grade 1 1 
No. % of 

total 


5 STUDENTS F( 
Grade 12 
No. % of 

total 


3R 1998-1999 
Grade 13 

No. % of 

total 


Grade 14 
No. % of 

total 


TOTAL 


























Gender 


Male 


























Female 


























1 , Ethnicity 


White 


























Black 


























Hispanic " 


'i 
























Asian 


- 
























Nati.e- 

Americar. 


























£\her 


























Special Population* 


ESE 

students 












1 















*Please provide definition of special population students used by your consortium. 
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4. The table below requests a breakdown of the general population of the consortium that is 
eligible to participate in Tech Prep. Answer the following questions and provide the 
information to complete the following table: 



• How do you define an eligible student? 



DEMOGRA 


PHIC CHAR 
Grade 9 
No. %of 
total 


ACTERISTICS 
Grade 10 
No. % of 

total 


OF THE GENE 
Grade 1 1 
No. % of 

total 


RAL POPULAR 
Grade 12 
No. %of 

total 


ION FOR 1998-1 
Grade 13 
No. % of 

total 


999 

Grade 14 
No. % of 

total 


TOTAL 


























Gender 


Male 


























Female 


























Ethnicity 


White 


























Black 


























Ffispanic 


























Asian 


























Native- 

American 


























Other 


























Special Population* 


ESE 

students 



























*The definition of special population should be the same as the definition for the table in number 3. 



5. Review the data above. If there are areas that significantly differ from the demographic 
information in number 3, what might be the cause(s) for these differences? 
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6. Please check (V ) those services provided at each grade level to assist students in the selection 
of and/or preparation for participation in appropriate Tech Prep programs of study: 



Services Provided 




Grade 

7 


Grade 

8 


Grade 

9 


Grade 

10 


Grade 

11 


Grade 

12 


Grade 

13 


Grade 

14 


Career Gui dance/C ounseling 


















Career/Occupational 
Assessment/Test mg 


















Career Exploration Activities 



















7. Check ( ■/ ) the student assessment measures other that your consortium is using or plans to 
use: 



Tech Prep Outcomes 


Currently Using 


Planning to 
Use 








Student enrollment 






Attendance 






Graduation rates 






Technical competency attainment 






Student attitudes/satisfaction 






Employer feedback/satisfaction 






Program of study continuation rates 






Program of study completion rates 






Academic gains 






Job placement 






Parental satisfaction 






Teacher Feedback 






Standardized assessment test(s) Please list: 






Other: (Please explain) 






Other: (Please explain) 






Other: (Please explain) 
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8. Do you have any evidence that Tech Prep students' performance in mathematics, science or 

communications has improved as a result of integrated instruction, contextual academics or 
other Tech Prep initiatives? Yes No 

If yes, please cite or attach documentation. 

9. Have you assessed Tech Prep student perceptions of Tech Prep? Yes No 

If yes, please summarize findings and describe method(s) you used to assess: (attach 
instrument or survey form if used.) 



10. Have you assessed Tech Prep student families' perceptions? Yes No 

If yes, please summarize findings and describe methods(s) you used to assess (attach 
instrument or survey form if used.) 



11. Please describe the strategies your consortium is using (will use) to place Tech Prep 
completers in above-minimum wage jobs related to their area of preparation. 



12. What strategies does your consortium employ to help students develop the attitude that 
education is a lifelong activity and that graduation and employment are milestones rather 
than the end of their education? 

13. Describe the major area(s) of impact that the implementation of Tech Prep is having within 
your consortium. 



14. What issues need to be addressed from the state and/or local level to enhance the 
implementation of Tech Prep within your consortium? 
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15. Are Tech Prep students taking advantage of the developed articulation agreements that 
support your consortium's Tech Prep Programs of Study? If so, please give examples? If 
not, why? 

Yes No 

16. Please provide the following information on formally signed articulation agreements and 
Tech Prep Programs of Study within your consortium. 

A. Number of formally signed articulation agreements 

B. Number of Programs of Study that have been developed and supported by these 

articulation agreements . 

C. Please provide a listing of the Program of Study titles that have been developed. 



Estimate the number and percentage of Tech Prep students who have taken the following 
Courses in 1998-1999. 




# of TP 
Students 


% Female 


% Male 


Applied Math I 








Applied Math II 








Algebra I 








Algebra II 








Geometry 








Applied Communications 








Applied Biology /Chemistry 








Chemistry (Lab) 








Biology (Lab) 








Principles of Technology I 








Principles of Technology II 








Physics 








Other grade level or above courses 
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PLANNING, DEVELOPMENT, AND IMPLEMENTATION INDICATORS 

For each of the following statements, please estimate , using the accompanying scale, the 
extent to which each suggested indicator is currently visible/evident within your 
consortium. For each statement, please add any other indicators that are relevant for your 
consortium. 



0 = 0 % 1 = 1 % - 25 % 2 = 26 % - 50 % 3 = 51 % - 75 % 4 = 76 % - 100 % 



1. 



0 

0 

0 

0 

0 

0 

0 

0 



Higher career 

12 3 4 

12 3 4 

12 3 4 

12 3 4 

12 3 4 

12 3 4 

12 3 4 

12 3 4 



expectations have been established for Tech Prep students. 

Suggested Indicators: 

a. Students perceive that higher expectations have been established for 
students enrolled in a Tech Prep program of study. 

b. Teachers perceive that higher expectations have been established for 
students enrolled in a Tech Prep program of study. 

c. Tech Prep students are taking higher level academic courses in 
mathematics, science, and communications than did general education 
students in previous years. 

d. Teachers have positive views about students' ability to achieve higher, 
expectations. 

e. Vocational-technical teachers stress their mission of preparing students 
for employment and further education. 

f. Academic teachers stress their mission of preparing students for 
employment and further education. 

g. Participating schools in the consortium have a special admissions 
process to Tech Prep that makes students feel distinct. 

h. Other: 



0 = 0% 1=1 % - 25% 2 = 26% - 50% 3 = 51%- 75% = 76% - 100% 



2. Students are pursuing technical/academic studies required to complete a Tech Prep 
program of study at the secondary level that includes an appropriate number of credits 
in vocational-technical courses and an appropriate number of credits in career-related 
applied mathematics and science or other mathematics and science courses at or above 
grade level. 



0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 



Suggested Indicators: 

a. More students take at least two mathematics and two science courses 
above basic or general courses. 

b. Students report that their vocational-technical teachers are encouraging 
them to take appropriate mathematics and science courses. 

c. Students report that their academic teachers are utilizing applied 
(active-learning) methodologies to teach mathematics, science, 
communications and technology courses in Tech Prep curricula. 

d. Other: 



0 = 0% 1 = 1% -25% 2 = 26% -50% 3 = 51% -75% 4 = 76% - 100% 



National Research Center for Career and Technical Education 



87 



Measuring Tech Prep Excellence: A Practitioner's Guide to Evaluation 



3. 



0 

0 

0 

0 

0 



4. 



0 

0 

0 

0 

0 



5. 



0 

0 

0 



Vocational-technical courses have been revised to include the basic mathematics, 
science, and communications competencies that underlie each occupational field of 
study. 



12 3 4 

12 3 4 

12 3 4 

12 3 4 

12 3 4 



Suggested Indicators: 

a. There is an increase in time and emphasis devoted to competency 
instruction. 

b. Academic instructional materials have been developed/adopted by 
academic teachers to emphasize mathematics, science, and 
communications skills as they relate to occupational fields or clusters. 

c. Vocational materials have been developed/adopted by vocational- 
technical teachers to emphasize mathematics, science, and 
communications skills as they relate to occupational fields or clusters. 

d. Instructional strategies are being used that stress reading and writing in 
the technical field. 

e. Other: 



1 = 1% -25% 2 = 26% -50% 3 = 51% -75% 4 = 76% -100% 



0 = 0 % 



Vocational-technical and academic teachers are working together to coordinate and 

integrate academic and vocational-technical education. 

Suggested Indicators: 

1 2 3 4 a. Increased number of joint planning sessions among vocational- 

technical and academic teachers. 

1 2 3 4 b. Increased number of demonstrated integrated teaching activities among 

vocational-technical and academic teachers at the secondary level. 

1 2 3 4 c. Increased number of demonstrated integrated teaching activities among 

vocational-technical and academic teachers at the postsecondarv level. 

1 2 3 4 d. Secondary and postsecondary teachers meet to coordinate curriculum 

development activities. 

1 2 3 4 e. Other: 



0 = 0 % 



1 = 1% -25% 2 = 26% -50% 3 = 51% -75% 4 = 76% -100% 



Tech Prep students are being assisted through career assessment and counseling to 
develop a four-year (2+2) or six-year (4+2) educational plan that will lead to successful 
employment and/or further study. 

Suggested Indicators: 

1 2 3 4 a. Individualized educational plans for Tech Prep students are developed 

and used. 

1 2 3 4 b. Individualized Tech Prep student educational plans reveal that 

students= programs of study match their perceived goals. 

1 2 3 4 c. Individualized Tech Prep student educational plans reveal that students 

are taking a sequence of mathematics, science, and communications 
courses that is appropriate for their chosen occupational or technical 
field. 
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0 12 3 4 



0 12 3 4 



0 12 3 4 



0 = 0 % 



0 

0 



d. All students are given an aptitude and occupational and/or technical 
assessment test and are assisted in interpreting the results. 

e. Counselors demonstrate an understanding of Tech Prep and are 
recruiting students. 

f. Other: 

1 = 1 % - 25 % 2 = 26 % - 50 % 3 = 51 % - 75 % 4 = 76 % - 100 % 



0 12 3 4 



0 12 3 4 



At risk and special population Tech Prep students are identified and are provided 
services to prepare them to succeed in their academic and vocational-technical courses. 

Suggested Indicators: 

a. Students indicate they receive special and extra help from 

mathematics, science, communications and vocational-technical 
teachers. 

Minimum reading and mathematics achievement levels are established 
to identify students needing extra help. 

Tech Prep admission process identifies students needing special help. 
Individualized educational plans are on file for students with basic 
skills deficiencies. 

Teachers and students verify coordination of vocational-technical, 
mathematics, science, and communications instruction for students 
with deficiencies. 

Special instruction is provided to students with mathematics, science, 
or communications deficiencies. 

Other: 



2 

2 



3 

3 



4 

4 



0 12 3 4 



0 12 3 4 



b. 

c. 

d. 

e. 

f. 



0 12 3 4 



g- 



0 = 0 % 



1 = 1 % - 25 % 2 = 26 % - 50 % 3 = 51 % - 75 % 4 = 76 % - 100 % 



7 . 



Professional development to reinforce mathematics, science, and communications 
competencies as an integral part of vocational-technical instruction is provided to 
Vocational-Technical and Academic teachers. 

Suggested Indicators: 

a. Recent staff development or college credit instruction has been 
provided in applied teaching strategies for mathematics, science, and 
communications skills underlying the vocational-technical field. 

b. Curriculum and instructional plans and materials reflect mathematics, 
science, and communications competencies that underlie the 
vocational-technical field. 

c. Classroom observation reveals that vocational-technical teachers teach 
and reinforce mathematics, science, and communications competencies 
that underlie their vocational-technical field. 

d. Classroom observation reveals that academic teachers teach and 
reinforce academic competencies as they relate to appropriate 
occupational fields. 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 = 0 % 



1 = 1 % - 25 % 2 = 26 % - 50 % 3 = 51 % - 75 % 4 = 76 % - 100 % 
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0 

0 

0 



8 . 



0 

0 

0 

0 

0 



9 . 



0 

0 



0 

0 

0 



12 3 4 
12 3 4 
12 3 4 



e. Vocational-technical and academic faculty are given extended periods 
of time to leam from each other. 

f. Vocational-technical and academic faculty share joint professional 
development activities. 

g. Other: 



0 = 0 % 1 = 1 % - 25 % 2 = 26 % - 50 % 3 = 51 % - 75 % 4 = 76 % - 100 % 



Vocational-technical and academic teachers and guidance counselors receive staff 
development to help students learn mathematics, science and communications concepts 
through familiar, tangible, and direct experience related to students’ occupational/ 
technical field of study. 

Suggested Indicators: 

1 2 3 4 a. Students and teachers perceive that more time is being devoted to 

applied teaching and more real-life examples are used to associate 
competencies with vocational-technical applications. 

1 2 3 4 b. Vocational-technical and academic teachers have received recent staff 

development on applied instructional methods, effective teaching, and 
application of competencies. 

c. Curriculum and instructional plans reflect applied teaching strategies. 

d. Classroom observation reveals increased use of contextual teaching 
methods for students following a Tech Prep program of study. 

1 2 3 4 e. Student handbooks and/or other student materials at the secondary and 

post-secondary level contain information on offered Tech Prep 
programs of study offered by the consortium. 



2 

2 



4 

4 



0 = 0 % 



1 = 1 % - 25 % 2 = 26 % - 50 % 3 = 51 % - 75 % 4 = 76 % - 100 % 



Students= successful completion of a Tech Prep program of study is followed by further 
study and/or placement in an above-minimum wage job in the field for which they have 
been prepared. 



12 3 4 

12 3 4 

12 3 4 

12 3 4 

12 3 4 



Suggested Indicators: 

a. Regional occupational forecasts at the time the students enter Tech 
Prep indicate that above-minimum wage jobs in their field of study 
will be available. 

b. Businesses and industries participate in the periodic review and up- 
dating of programs of study and curricula to ensure the 
occupational/technical material taught is relevant for the current job 
market. 

c. Businesses and industries are actively sought to provide internships/ 
apprenticeships/mentors to students to give them experience in their 
field prior to placement. 

d. Tech Prep completers are employed in jobs for which they were 
educated. 

e. Tech Prep completers are satisfied with their jobs. 
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0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 



f. Employers of Tech Prep completers are satisfied with their job 
performance. 

g. Tech Prep completers salaries are higher than minimum wage 
earnings. 

h. Tech Prep completers exhibit the attitude that to be productive citizens 
in a competitive workforce they must continue to learn throughout 
their lives. 

i. Tech Prep completers indicate they have been adequately prepared to 
continue their education by acquiring lifelong learning skills. 

j. Other: 



0 = 0 % 1 = 1 % - 25 % 2 = 26 % - 50 % 3 = 51 % - 75 % 4 = 76 % - 100 % 



V. TECH PREP BEST PRACTICES (Optional) 

The demonstration and showcasing of statewide best practices was identified as one of the goals 
and recommendations by the Millennium Task Force in their report to the Commissioner of 
Education. Your assistance is requested in identifying Best Practices in curriculum and 
instruction, guidance and counseling, and business partnerships for your consortium. Using the 
templates, please solicit Best Practices from the administrators, teachers and guidance staff in 
your consortium. Selected Best Practices will be input on the web site. There are samples of 
Best Practices located on the Institute for Workforce Competitiveness' (IWC) web site: 
www.fiu.edu/~xiwc. The templates provided are also located on the IWC's web site. Upon 
review, the best practices submitted will also be available on the Internet. 



Thank you for completing this report. Please return the completed annual report, either by mail 
or e-mail, to: 

Institute for Workforce Competitiveness 
Florida International University 
University Park, EAS - 2613 
Miami, Florida 33199 
byrdj@ fiu.edu 



Permission to reprint material obtained from F. Hammons, Associate Professor, Florida 
International University, FL. 
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APPENDIX F 

Tech Prep Evaluation System for Illinois (TPESI) (IL) 

School Assessment Form 

Name of School Visited: 

Name of Team Member: 

Date of Visit: 

This school assessment form will be used by each team member to rate his/her assessment of the 
implementation stage and quality of the essential and supporting elements of Tech Prep. 

Scales: 

• Implementation Stage: 1- Not Begun, 2-Planning, 3-Development, and 4-Implementation. 

• Quality: 1 -Poor, 2-Fair, 3-Good, and 4-Excellent. 



Elements 


Implementation Stage 


Quality 


1 


2 


3 


4 


1 


2 


3 


4 


Example: E4. Inservices for Teachers 






X 










X 


El. 2+2 Program 


















E2. Articulation 


















E3. Curriculum Development 


















E4. Inservice for Teachers 


















E5. Inservice for Counselors 


















E6. Equal Access for Special Pops 


















E7. Preparatory Services 


















E8. Work- based Learning 


















SI. Leadership, Commitment, and 
Administrative Support 


















S2. Parental Support 


















S3 . Business/Labor/Community Involvement 


















S4. Transition of Students to Postsecondary 
Education 


















S5. Identification and Accurate Reporting of 
Tech Prep Students 


















S6. Secondary and Postsecondary Collaboration 


















S7. Evaluation and Program Improvement 


















S8. Integrated, Contextual Instruction Strategies 
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Consortium Assessment Form 



Title of the Consortium: 

Name of Team Member: 

Date of Site Visit: 

This consortium assessment form is a tool for assessing essential and supporting elements of 
Tech Prep for the overall consortium. The consortium assessment form is composed of two 
sections: (1) essential elements, and (2) supporting elements. Based on individual team 
member's assessment of the consortium overall, the team leader should provide the team's overall 
assessment. 

TECH PREP ELEMENTS 



Please assess the stage of implementation and quality of each element following the directions below. 

Staee of Implementation : Use the following scale to indicate the stage of implementation for each 
essential element of Tech Prep: 



l=Not begun 


This stage indicates the element has not yet begun to be implemented. 


2=Planning 


This stage includes goal setting, orientation, the formation of committees 
and teams, and the development of plans related to a Tech Prep element. 


3=Development 


This stage involves designing, creating, and field testing a specific 
element. 


4- Ini pie mentation 


This stage occurs when plans, activities, and/or products associated with a 
specific element are being carried out. 



Rationale: Substantiate your rating on the implementation of each element. 



Quality of Elements : Use the following scale to indicate the quality of each element and provide a 
rationale for your assessment. 



1. Poor 2. Fair 3. Good 4. Excellent 



Rationale: Substantiate your rating on the quality of each element. 

Additional Comments and Recommendations : Provide additional comments on the element such as an 
exemplary case and recommendations for program improvement. 
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Essential Element #1: 2 + 2 Program That Leads to Associate Degree - All Tech Prep 
programs must have a sequence of appropriate advanced academic and technical courses. 

Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 

Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 



l.Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 



National Research Center for Career and Technical Education 
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Essential Element #1: 2+2 Program That Leads to Associate Degree 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Comprehensiveness within the 
program - Tech Prep programs 
and courses reflect all aspects of 
industry. 


# and % all Tech Prep of 
programs and courses 
reflecting all aspects of 
industry 


Tech Prep student perceptions of the 
comprehensiveness of the programs 


Relevance to business area 
industry demands - Tech Prep 
programs and courses address 
competencies and standards 
required by business and industry. 
Tech Prep programs target career 
areas that are in greatest demand 
in the region. 


# and % all Tech Pep of 
programs matched with 
area business/industry 


# and % of students who found jobs 
related to their Tech prep programs 


Rigorous coursework and 
advanced skills - Tech Prep 
programs and courses demand 
advanced academic and technical 
skills to be accepted by 
postsecondary institutions. Tech 
Prep programs include courses 
using advanced technology. 


# and % all of Tech Prep 
programs offering high- 
wage jobs 

# and % of all Tech Prep 
program leading to 
advanced placement and 
4-year colleges 


# and % of Tech Prep graduates getting 
high- wage jobs 

# and % of high school Tech Prep 
students matriculated 2-year/ 4-year 
colleges 

# and % of Tech Prep students taking at 
least 3 years each of math and science 

# and % of Tech Prep students in high 
school and college taking advanced 
courses 


Appropriate sequential 
arrangement - Tech Prep 
programs and courses have 
sequential alignment between 
secondary and postsecondary 
education, resulting in seamless 
2+2 paths for Tech Prep students. 


# and % of all Tech Prep 
programs offering 
complete sequential 2+2 
paths 


# and % of Tech Prep students taking 
remedial classes and credits when 
entering college 


Flexibility of Tech Prep 
programs to the local 
educational context - Tech Prep 
programs are arranged to 
encompass various models 
including 2+2, 2+4, 2+2+2 to 
adapt to local situations. 


# and % of all Tech Prep 
programs that offer 2+2, 
2+4, and other models 


# and % of Tech Prep students who 
participate in each of the models 
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Essential Element # 2 : Articulation - All Tech Prep programs must be carried out under an 
articulation agreement(s) between secondary and postsecondary institutions and a process must 
be in place to follow students from secondary Tech Prep programs to postsecondary Tech Prep 
programs^ 



Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 

1. Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 



National Research Center for Career and Technical Education 
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Essential Element #2: Articulation 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Comprehensiveness of 
articulation agreements - 

Articulation agreements cover 
academic and technical courses. 


# and % of articulated 
programs/courses of total 
program courses 


# and % of Tech Prep students taking 
articulated academic and technical 
programs (courses) 


Timeliness/on-going update - 

Articulation agreements are 
updated and documented on an 
on-going basis to reflect changes 
in program sequences between 
secondary and postsecondary 
education. 


Frequency of articulation 
agreement updates, 
whether articulation 
agreements reflect current 
practice 


# and % of Tech Prep students have 
access to up-to-date articulation 
agreements 


Advancement - Articulation 
agreements are available in 
advanced academic and 
technical courses, encompassing 
advanced placement and dual 
credit courses, resulting in 
students' smooth transition to 
postsecondary education. 


# and % of all 
agreements that contain 
advanced courses 


# and % of Tech Prep students taking 
advanced courses 

# and % of Tech Prep students entering 
postsecondary institutions through 
articulation agreement 


Widespread information 
sharing - Information about the 
contents of articulation 
agreements, including dual- 
credit, is well communicated 
within the consortium. 


# and % of all faculty, 
teacher, parents, students 
who are aware of 
articulation agreements 

Student perception about 
information about 
articulation 


Tech Prep students know about 
articulation program/course options 
(student awareness) 
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Essential Element # 3: Curriculum Development - All Tech Prep program curricula must be 
aligned with the Illinois Learning Standards, Occupational Skill Standards (where available), 
Workplace Skills and Career Development Competencies. This curriculum must include 
applied methodology that integrates academic and technical coursework. 



Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 



1. Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 
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Essential Element #3: Curriculum Development 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Curriculum alignment with 
standards - Tech Prep programs 
and courses are aligned with 
Illinois Learning Standards and 
incorporate employability skills 
and occupational skill standards. 


# of courses that reflect 
ILS, OSL, WS, and CDC 


# and % of Tech Prep students who 
took the courses, and their performance 
on academic, occupational and 
employability skills assessments 

# and % of Tech Prep students who 
meet the Illinois Learning Standards 


Curriculum integration 
between technical and 
academic education - 

Academic courses are being 
changed to incorporate career 
related contents and technical 
courses, and incorporate contents 
demanding more advanced 
academic skills. Curriculum 
integration involves a broad 
range of skills including 
academic, technical skills and 
career-related information. 


# of academic and 
technical courses that 
incorporate integrated 
curriculum and 
instructional strategies 


# and % of all students receiving classes 
with career-related curriculum 

# and % of all Tech Prep students 
receiving an integrated curriculum 

Tech Prep student performance on the 
ISAT exam 


Curriculum change at the both 
secondary and postsecondary 
levels - Curriculum 
development is actively done at 
secondary and postsecondary 
schools. 


# of courses that have 
been developed as a 
result of Tech Prep 


College/HS student perceptions of the 
curriculum 

# and % of Tech Prep students who 
advanced to Community College Tech 
Prep programs 


Applicability of classroom 
instruction to the work place - 

Curriculum development 
incorporates hands-on and real 
world examples for students to 
apply what they leam in 
classroom to the actual work 
setting. Classroom instructions 
employ diverse delivery methods 
to enhance students’ contextual 
learning. 


# and % of all Tech Prep 
classes that use hands-on 
approaches 


Student perception of the applicability 
and relevance of courses 


Use of technology - Curriculum 
includes the use of technology in 
the delivery of instruction (i.e., 
distance education, websites). 


# and % of all Tech Prep 
classes that use 
technology 


Student familiarity with and perceptions 
of technology applications used in Tech 
Prep 
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Essential Element # 4: Inservice Training for Teachers - All Tech Prep projects must include 
inservice training for secondary and postsecondary instructors. 

Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1.) 

1. Not Begun 2. Planning 3. Development 4. Implementation 

Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 



1. Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 
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Essential Element #4: Inservice Training for Teachers 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Balance of content - Content of 
professional development for 
teacher covers a sufficient range 
of knowledge, skills, and 
abilities related to Tech Prep 
(i.e., activity doesn't try to cover 
too many topics/materials). 
Content is appropriate for 
secondary, postsecondary, 
academic and technical 
participants. 


Breadth of topics 
Depth of topics 

# and % of all secondary 
and postsecondary faculty 
participating participation 

# and % of all academic 
and technical faculty 
participating 




Satisfaction with activity - 

Type of activity is appropriate to 
need. Design, organization, and 
delivery of activity appropriate 
for content covered. 


Clear objectives 

# and % of participants 
who show a high level of 
satisfaction related to 
delivery format, methods, 
organization, etc. 

Allows for collaboration 
and dialogue 




Relevance of topic to practice - 

Content of activity is aligned 
with individual participant 
needs. Current and up-to-date 
topics covered in activity. 


# and % of participants 
who show a high level of 
satisfaction related to 
delivery format, methods, 
organization, etc. 

Level of satisfaction 
related to content 

Applicability of content 
to practice 


# and % of Tech Prep students who 
show a high level of satisfaction with 
changes in curriculum 

# and % of Tech Prep students who 
show a high level of satisfaction with 
changes in instructional strategies 

# and % of Tech Prep students who 
show a high level of knowledge of 
career clusters 


Accessibility of activities - 

Professional development 
activities are accessible to 
participants. 


Frequency of activities 
Location of activities 





102 



National Research Center for Career and Technical Education 



Measuring Tech Prep Excellence: A Practitioner's Guide to Evaluation 



Essential Element # 5: Inservice Training for Counselors - All Tech Prep projects must 
include inservice training for counselors. 

Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 

Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 

1 . Poor 2. Fair 3. Good 4. Excellent 

Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 



National Research Center for Career and Technical Education 
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Essential Element #5: Inservice Training for Counselors 



Dim ension/Desc ription 


Program 

Outcome/Measures 


Student Outcome/Measures 


Balance of content - Content of 
professional development for 
counselors covers a sufficient 
range of knowledge, skills, and 
abilities related to Tech Prep 
(i.e., activity doesn't try to cover 
too many topics/materials). 
Content is appropriate for 
secondary, postsecondary, 
academic and technical 
participants. 


Breadth of topics 

Depth of topics 

# and % of all secondary 
and postsecondary faculty 
participating participation 

Academic # and % of all 
technical faculty 
participating 




Satisfaction with activity - 

Type of activity is appropriate to 
need. Design, organization, and 
delivery of activity appropriate 
for content covered. 


Clear objectives 

# and % of participants 
who show a high level of 
satisfaction related to 
delivery format, methods, 
organization, etc. 

Allows for collaboration 
and dialogue 




Relevance of topic to practice - 

Content of activity is aligned 
with individual participant 
needs. Current and up-to-date 
topics covered in activity. 


# and % of participants 
who show a high level of 
satisfaction related to 
delivery format, methods, 
organization, etc. 

Level of satisfaction 
related to content 

Applicability of content 
to practice 


# and % of Tech Prep students who 
show a high level of satisfaction with 
changes in curriculum 

# and % of Tech Prep students who 
show a high level of satisfaction with 
changes in instructional strategies 

# and % of Tech Prep students who 
show a high level of knowledge of 
career clusters 


Accessibility of activities - 

Professional development 
activities are accessible to 
participants. 


Frequency of activities 
Location of activities 
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Essential Element # 6: Equal Access for Special Populations - All Tech Prep projects must 
provide equal access to the full range of Tech Prep education programs to individuals who are 
members of special populations 



Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 
l.Poor 2. Fair 3. Good 4. Excellent 

Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 
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Essential Element #6: Equal Access for Special Populations 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Equal access - All Tech Prep 
programs provide equal access 
for students in special 
populations. 


Existence of equal access 
policy within the 
consortium and each 
school and college. 


# and % of special population students 
in the programs 

# and % of Tech Prep students with 
special needs who report no exclusion 
from Tech Prep activities 


Student satisfaction - Students 
with special needs report 
inclusion in Tech Prep activities 
and believe their needs were 
accommodated. 




Students with special needs report no 
exclusion from Tech Prep activities 


Existence of accommodations 
and support services - Students 
with special needs are provided 
with adaptations, 
accommodations and other 
services that facilitate their 
successful participation in Tech 
Prep activities. 


Number and type of 
accommodations, 
adaptations and supports 
provide with in the 
consortium and each 
school and college 


# and % of all students who use the 
system and services 
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Essential Element # 7: Preparatory Services (including career guidance) - All Tech Prep 
projects must provide preparatory services that assist all populations to participate in Tech Prep 
education programs. Preparatory services may take the form of career development and 
guidance activities and recruitment. Tech Prep will include activities to ensure that Tech Prep 
students have an opportunity to become aware of the full range of career options, will be able to 
make an informed career choices, and will be provided with opportunities to select a career 
focus. Individualized Career Plans (ICPs) are required in order to be a Tech Prep student 

Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 



1. Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 
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Essential Element #7: Preparatory Services 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Reflecting the needs of students 

- Preparatory services and career 
guidance reflect the needs and 
development of the students and 
tailors its methods to the students’ 
needs and stage of development. 


Number and types of 
methods used 


Student acquisition of information 
about preparatory service 

Student satisfaction with preparatory 
services 


Diverse exposure to career and 
educational options - 

Preparatory services employ 
diverse methods and career 
exploration experiences to expose 
students to many different career 
areas. 


Number and types of 

methods used regarding: 

• Various career 
exploration 
experiences 

• Introduction to 
different career areas 

• 


Student acquisition of information 
about preparatory service. 

Student satisfaction with preparatory 
services 


Effectiveness of recruitment 
strategies - Students are aware 
of various preparatory services 
and of various Tech Prep 
activities and courses. 


Different types of 
recruitment strategies 


# and % of all students who are aware 
and satisfied with Tech Prep options 


Systematic developmental 
preparatory services - An 
organized system of Tech Prep 
services is offered (e.g. checklists, 
portfolios) tailoring its 
approaches to assessment, 
monitoring students' progress, 
career and academic guidance and 
recruitment according to students’ 
grade level. 


The structure of 
preparatory service 
delivery facilities 
advancement 


# and % of Tech Prep students 
reporting use of portfolios, 
assessments, ICP's, regularly 
scheduled meetings with teacher, etc 


Effective counseling and career 
development/decision making - 

Counseling helps students in 
selecting courses and making 
career decisions. 




Student perception about the 
effectiveness of counseling. 
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Essential Element # 8: Work-Based Learning Experience - All Tech Prep programs will 
include a work-based learning experience for students. Work-based learning may include career 
development experiences or skill development experiences. (Includes providing quality work- 
based learning for Tech Prep students, defined in the FY01 guidelines as “activities to improve 
methods of coordinating work-based learning with school-based learning. These activities 
should include curriculum coordination using Occupation Skill Standards, school-based mentor 
selection and training and work-based mentor selection and training.”) 

Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 

1. Poor 2. Fair 3. Good 4. Excellent 

Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 
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Essential Element #8: Work-Based Learning Experience 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


School-based learning and 
work-based learning (WBL) 
experiences connected - The 

WBL experiences are closely 
connected to the students' 
coursework and pertinent to the 
contents of school curriculum. 


Number of WBL sites 
associated with each Tech 
Prep program 


• and % of Tech Prep students 
participating in WBL 

• By students' own selection, and 

• Through school coordination 


Student satisfaction - Students 
believe that the WBL 
experiences enhance their 
learning and are helpful in 
making career choices. 




Student satisfaction with WBL 

Student perception regarding 
usefulness of WBL 

Student achievement associated WBL 
in terms of: 

• Academic achievement 

• Employment 

• Technical achievement 


Business/Industry/Labor 
support (B/I/Ls) - Area B/I/Ls 
provide opportunities for WBL 
experiences, facilitate learning 
opportunities, and provide 
financial support. 


Number of B/I/Ls that 
provided WBL to students 

Number of mentors within 
B/I/Ls who are actively 
engaged. 

Evidence of financial 
support 


Number of students who received 
awards and support from B/I/Ls. 

# and % of Tech Prep students who 
have worksite mentors 


Variety of WBL experiences - 

The WBL are provided in 
diverse forms such as job 
shadowing, internships and 
apprenticeships and are offered 
in different career clusters. 


Different types of WBL 
experiences associated 
with Tech Prep programs 


# and % of Tech Prep students who 
have different types of WBL 
experiences 


Student motivation for 
learning - The WBL 
experiences motivate students to 
work harder in school and to 
have greater interest n their 
studies. 




# and % of Tech Prep students who say 
that they are motivated by participating 
in WBL. 

Student achievement in coursework 
(GPA) 
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Supporting Element #1: Leadership, Organization and Administrative Support - Local 
administrators are actively engaged in ongoing implementation and improvement of the local 
Tech Prep initiative. Faculty, staff, and administrators are committed to and actively engaged in 
facilitating the continued growth, development and evolution for the Tech Prep initiative 



Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 



1 . Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 
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Supporting Element #1: Leadership, Organization and Administrative Support 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Participation and support - 

Administrators participate in 
Tech Prep actively and show 
leadership in implementing Tech 
prep. 


# and % of administrators 
who participated in Tech 
Prep meeting. 

Financial resources 
dedicated to Tech Prep 
key leaders. 


Student awareness of leadership 
support for Tech Prep. 

Student awareness of administrative, 
counselor, and parental support for 
Tech Prep. 


Create faculty/parental buy-in 

- Through the administrative 
leadership, faculty, teacher and 
parents show buy-in for Tech 
Prep and participate actively in 
it. 


# and % of faculty who 
participated in Tech Prep 
meeting, Professional 
Development, Curriculum 
Development and other 
Tech Prep events. 


Students perceive a high level of 
support for Tech Prep within then- 
schools and colleges. 


Supports collaboration 
among key people - Key 

personnel including the 
consortium coordinator, school 
Tech Prep team leaders, EFE 
directors, etc. maintain favorable 
working relationships in 
implementing Tech Prep, 
including sharing resources to 
maximize Tech Prep 
implementation. 


Level of collaboration 
between secondary and 
postsecondary 
administrators, Tech Prep 
coordinators and other key 
personnel, perceived by 
administrators, faculty, 
and counselors to be high. 


Students show a high level of 
awareness of collaborations to support 
Tech Prep programs and activities 
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Supporting Element #2: Parental Support - All Tech Prep consortia must provide information 
and services to parents for them to understand every aspect of Tech Prep. 

Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 

Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 



1 . Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 
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Supporting Element #2: Parental Support 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Parental awareness of Tech 
Prep - Parents are well aware of 
Tech Prep concept and Tech 
Prep program options for their 
children. Parents are actively 
involved in conversations about 
students' progress. 


# and % of parents who 
are aware of Tech Prep 


Students are highly informed 

Student satisfaction with the program 
is high 


Parental involvement in Tech 
Prep activities - Parents are 
actively involved in Tech Prep 
activities, including professional 
development and career 
guidance and counseling for 
their children. 


# and % of parents who 
are involved in various 
activities 


Students are highly informed. 

Student satisfaction with the program 
is high 


Parent attitude toward Tech 
Prep - Parents in the consortium 
show positive attitudes toward 
Tech Prep and support Tech 
Prep programs of study and 
related activities. 


Faculty perceive that of 
parental support for Tech 
Prep is strong 


Students are highly informed. 

Student satisfaction with the program 
is high 
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Supporting Element #3: Business/Labor/Community Involvement - Tech Prep program 
should include business, industry, labor, and community involvement, encompassing school- and 
work-based learning activities and informing curriculum change. 



Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 



1. Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 
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Supporting Element #3: Business/Labor/Community Involvement 



Dimension/Description 


Program 

Outcome/Measures 


Student 

Outcome/Measures 


Extent of 

Business/Labor/Community 
(BLC) Involvement - BLC 
involvement is based on 
collaborative efforts, a high level of 
input, and the central role played by 
BLC in the consortium. In addition, 
the BLC involvement is 
comprehensive across programs and 
visionary in nature (i.e., anticipating 
and driving future skill needs of the 
workforce). 


Visible ties with local 
employers 

BLC actively involved in 
driving necessary skill sets 

Established plans for review of 
necessary skills 

Financial Support provided 




Satisfaction of BLC - BLC take 
pride in participating. They view 
participation as a practical 
opportunity to recruit qualified 
students. Their expectations (as 
well as that of the consortium) are 
met regarding involvement. 


Clear objectives regarding BLC 
involvement. 

Level of satisfaction related to 
caliber of students they receive, 
collaboration with secondary 
and postsecondary faculty, and 
their own perception of 
involvement in the community 
(i.e., they are satisfied with their 
efforts in “giving back to the 
community”) 

View their involvement to be 
beneficial to students in the 
long-run 


Students perceive that they are 
receiving real world; applicable 
experiences while in the 
program. 


Relevance of BLC involvement - 

Involvement is program specific, 
centered around WBL experiences, 
and driven by labor market need for 
placing qualified graduates in the 
work place. In addition, BLC 
activities are aligned to all Tech 
Prep programs and the opportunities 
provided to students as well as 
faculty are current and up-to-date. 


Number of established WBL 
programs for students and 
faculty to gain industry 
experience that are aligned to 
their program of study (e.g. 
number of WBL programs for 
each Tech Prep program). 

Curriculum supports skills 
necessary for industry 


# and % of Tech Prep student 
who are involved in WBL 
experiences, job shadowing, etc 
related to their respective 
program areas 

Student satisfaction with their 
contact with BLC and the 
relevance of that contact 


Accessibility - BLC encourage 
student, faculty, and administrator 
presence in their work places 
through established programs (e.g., 
WBL and professional 
development). 


Frequency of activities (e.g., 
how often are students and 
faculty given access through 
such established programs?) 

Location of activities 

Participation rate 


Students perceive they have 
access to practical and 
meaningful work experiences 
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Supporting Element #4: Transition of Students to Postsecondary Education - Tech Prep 
includes activities to support development and implementation of a transition system for Tech 
Prep students that will significantly increase the number of high school graduates who are 
transitioning to postsecondary technical programs immediately following high school without 
remediation. 

Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 

1. Poor 2. Fair 3. Good 4. Excellent 

Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 
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Supporting Element #4: Transition of Students to Postsecondary Education 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Provides skills necessary for 
postsecondary education - 

Tech Prep coursework, 
integrated instruction and WBL 
experiences provide the 
necessary skills to support a 
successful transition to 
postsecondary education. 


Evidence of integrated 
SBL and WBL 
experiences 

Availability of advanced 
courses 

Coursework articulates 
toward college credit 


# and % of Tech Prep students who 
take remedial courses after 
matriculation to community college 

# and % of Tech Prep students who are 
college ready. 

# and % of students who go on to 
postsecondary education 


Facilitates students' 
aspirations for postsecondary 
education - Students are 
encouraged to continue their 
postsecondary education and are 
more focused on their career 
aspirations. 




# and % of Tech Prep graduates who 
attend a 4-year postsecondary 
institutions 

Students report being more focused 
and clear about their aspirations 


Availability of transition 

services - The consortium 
provides services to inform 
students of postsecondary 
options, career goals and majors, 
and academic skills and courses 
needed for a successful transition 


Number of services 
available to students 


# and % of Tech Prep students who use 
transition services 


Student tracking system - A 

tracking system is in place that 
gathers data about the transition 
outcomes of their graduates. 


Type of tracking system 

Use of the system by 
schools and consortium 

Evidence of a tracking 
system, e.g. follow-up data 
on graduates 




Facilitates students' confidence 
for their postsecondary 
education - Students are more 
likely to show confidence in 
pursuing postsecondary 
education. 




# and % of Tech Prep students who 
show confidence 
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Supporting Element #5: Secondary/Postsecondary Collaboration - Tech Prep initiatives 
should include strong collaborative administrative, faculty, and curricular connections between 
secondary and postsecondary institutions. These efforts should include the community college 
providing leadership in the ongoing implementation of the local Tech Prep initiative. 



Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 



1 . Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 



National Research Center for Career and Technical Education 



119 



Measuring Tech Prep Excellence: A Practitioner's Guide to Evaluation 



Supporting Element #5: Secondary and Postsecondary Collaboration 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Availability of joint 
participation - Program - 
program collaboration between 
secondary and postsecondary 
faculty. Collaboration provides 
opportunities to customize 
community needs, engage 
faculty and educators, and 
successfully transition students. 


Dual credit opportunities 

Positive relationships 
between secondary and 
post-secondary institutions 

Established professional 
development/faculty 
exchanges programs 

Information and facilities 
sharing 

Existing annual meetings 
for curriculum review 


# and % of Tech Prep students 
receiving dual credit (and who have 
option to receive dual credit) 


Perceived satisfaction - 

Faculty, staff, and students 
perceive strong working 
relationships between secondary 
and postsecondary institutions 
through established roles, 
smooth student transitions, and 
resolutions of shared concerns 


Strong ties between 
secondary and 
postsecondary faculty and 
staff 


Student perceptions of smooth 
transition and relevance of program 
content between High School and 
Community College 
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Supporting Element #6: Identification and Accurate Reporting of Tech Prep Students - 

Tech Prep will include activities to improve methods of identifying and reporting Tech Prep 
students at the high school and postsecondary levels based upon the State’s definition. 



Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 




Quality of Element : Provide your own assessment of the quality of the element. 



1. Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 




National Research Center for Career and Technical Education 



ERIC 
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Supporting Element #6: Identification and Accurate Reporting of Tech Prep Students 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Congruence of actual Tech 
Prep students with reported 
Tech Prep students - The 

counted and reported Tech Prep 
students are congruent with the 
actual Tech Prep students, 
according to state and local 
definition of Tech Prep programs 
of study. 


Congruence between 
perceived and reported 
Tech Prep students. 


# and % of all students participating in 
Tech Prep 

Enrollment trends over time 


Clear and Agreed upon 
Definition of Tech Prep 
students - Various groups of 
personnel including 
administrators, faculty, teacher, 
students, parents and employers 
are clear about how Tech Prep is 
defined and who is considered 
Tech Prep students. 


# and % of administrators, 
faculty, teacher, students 
who understand what Tech 
Prep is and who Tech Prep 
students are. 


Students are able to identify 
themselves as participants in Tech 
Prep programs and activities 


Reporting system - The 
consortium and schools within it 
have Tech Prep student reporting 
system and this system provide 
accurate Tech Prep student data. 
This system includes data base 
related Tech Prep courses or 
programs, students who 
take/took the courses, and other 
follow-up data from secondary 
to postsecondary to employment 
(including military) - same as 
student tracking system 


Existence of reporting 
system 


Students are able to identify 
themselves as participants in Tech 
Prep programs and activities 
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Supporting Element #7: Evaluation and Program Improvement - Tech Prep includes the 
systematic collection of information about the activities, characteristics, and outcomes of 
programs for use by specific people to reduce uncertainties, improve effectiveness, and make 
decisions with regard to what those programs are doing and affecting. 



Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 



1 . Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 



National Research Center for Career and Technical Education 



123 



Measuring Tech Prep Excellence: A Practitioner's Guide to Evaluation 



Supporting Element #7: Evaluation and Program Improvement 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Comprehensiveness of 
methods - Evaluation system 
utilizes at least one qualitative 
and one quantitative method of 
data collection. Process relies on 
evaluation team. 


# relevance of value 
qualitative methods 

# relevance of value 
quantitative methods 

Evaluation teams in place 
within consortium and 
schools and college 




Systematic - Methods of the 
evaluation system work together 
to tell the Tech Prep story. 


Evaluation design is such 
that one method of data 
collection builds upon a 
previous method(s) 

Evaluation plan in place 




Validity of system - Evaluation 
system measures what it claims 
to measure. 


Evaluation system answers 
critical evaluation 
questions 




Utilization of results - 

Evaluation system provides data 
that are beneficial to making 
decisions related to program 
improvement. 


Action plans in place 

Ongoing and meta 
evaluation used 

% of change, 
improvement, etc 
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Supporting Element #8: Integrated, Contextual Instructional Strategies - Tech Prep 
curriculum should include instructional strategies that are designed to meet the needs of all 
learners. Contextual learning, active learning, and problem-based instruction are examples of 
strategies that teachers should incorporate into school-based teaching to ensure that learning is 
more relevant and applicable to work, family and community life. 



Stage of Implementation : Provide your own assessment of the implementation stage of the element. 
(Please refer to the scale guidelines on page 1 .) 

1. Not Begun 2. Planning 3. Development 4. Implementation 



Provide a rationale for your assessment of implementation: 



Quality of Element : Provide your own assessment of the quality of the element. 



1 . Poor 2. Fair 3. Good 4. Excellent 



Provide a rationale for your assessment of quality: 



Additional Comments and Recommendations: 



National Research Center for Career and Technical Education 



125 



Measuring Tech Prep Excellence: A Practitioner's Guide to Evaluation 



Supporting Element #8: Integrated, Contextual Instructional Strategies 



Dimension/Description 


Program 

Outcome/Measures 


Student Outcome/Measures 


Makes connections to the real 

world - Integrated, contextual 
instruction makes connections 
between the real world 
(especially the world of work) 
and classroom instruction. 


# and % of academic and 
technical courses that 
reflect real-world 
applications 


# and % of Tech Prep students who 
enrolled in courses having real-world 
applications 

Tech Prep student ability to apply 
knowledge across diverse settings (e.g., 
school, work) 


Blends theory and practice - 

Integrated, contextual instruction 
creates meaningful linkages 
between theoretical and practical 
concepts. 


# and % of academic and 
technical courses that 
integrate theory and 
practice 


# and % of Tech Prep students who 
enrolled in courses having integrated 
theory and practice 

Tech Prep student ability to apply 
knowledge across diverse settings (e.g., 
school, work) 


Encourages individual but also 
team learning - Integrated, 
contextual learning encourages 
students to engage in 
collaborative learning situations 


# and % of academic and 
technical courses that 
engage students in team 
learning strategies 


# and % of Tech Prep students who 
enrolled in courses having students 
engage in team learning strategies 

Tech Prep student ability to participate 
in team work successfully 


Facilitates constructivist 
approaches to curriculum - 

Integrated, contextual instruction 
encourages students' active 
participation in the construction 
of knowledge in a variety of 
learning environments 
(classrooms, work settings, 
laboratories) 


# and % of academic and 
technical courses that 
facilitate constructivist 
approaches 


# and % of Tech Prep students 
participating in courses that facilitate 
constructivist approaches 

Tech Prep student ability to think 
critically, make meaningful decisions, 
solve problems 


Utilizes authentic, 
performance-based assessment 

- Integrated, contextual 
instruction focuses on assessing 
learner skills and knowledge 
using realistic tasks and settings. 


# and % of academic and 
technical courses that 
utilize authentic, 
performance-based 
assessment 


# and % of Tech Prep students who 
enrolled in courses utilizing authentic, 
performance-based assessments 

Tech Prep student ability to demonstrate 
skills and knowledge in authentic 
settings 



Permission to reprint material obtained from D. Bragg, Project Director, University of Illinois, 
Urbana/Champaign, IL. 
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APPENDIX G 

Tech Prep Consortium Site Evaluation (TX) 

Self-Study 

This self-study is designed to help Tech-Prep consortium staff prepare for the peer-review, site 

visit evaluation process. The completed self study should be completed in either Word or Word 

Perfect format and returned at least 14 calendar days before the scheduled site visit to: 

Rob Franks, Program Director for Tech-Prep 
Community and Technical Colleges Division 
Texas Higher Education Coordinating Board 
franksrt@thecb.state.tx.us 

Address any questions to Rob Franks at franksrt@thecb.state.tx.us or (512) 427-6442. 

Area 1 - The Consortium's Five-Year Tech-Prep Plan 

1. Is the Consortium in compliance with its approved five-year plan of operation? 
a) If not, in what areas and why? 

2. In what areas does the consortium excel in meeting goals and objectives of the plan? 

Area 2 - Annual Curriculum Alignment Meetings 

3. Do members of the Tech-Prep consortium and/or consortium staff facilitate annual 
subject-area articulation meetings between secondary and post-secondary faculty and 
administrators? 

4. If yes, how often are these meetings held? How are the results of these meetings 
documented? 

5. What process is used to disseminate the results of these meetings disseminated? Who 
receives them? 

6. Does the consortium have a process to track these alignments from paper documents to 
utilization in the classroom? 

Area 3 - Tech-Prep Six-Year Plans 

7. Who in your consortium is responsible for the development and dissemination of Tech- 
Prep six-year plans? 
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8. How are counselors in your consortium's schools and colleges utilizing state-approved, 
six-year Tech-Prep plans? 

9. Do you have examples of how these six-year plans are being used by other members of 
the consortium, such as Tech-Prep students? 

Area 4 - Identification and Reporting of Tech-Prep Students 

10. Describe the consortium's processes for the identification and reporting of Tech-Prep 
program participants. 

1 1 . Does the consortium sponsor training of public school and/or college staff in the correct 
identification and reporting of Tech-Prep students? Describe. 

Area 5 - Training Programs for Counselors 



12. What examples of training programs for counselors that the Consortium has provided can 
enable them to more effectively: 

a) Provide information to students regarding Tech-Prep programs; 

b) Support student progress in completing Tech-Prep programs; 

c) Provide information on related employment opportunities; 

d) Ensure that such students are placed in appropriate employment; and 

e) Stay current with the needs, expectations, and methods of business and all aspects of 
industry. 

Area 6 - Introduction of Tech-Prep Programs to Students 

13. When and how is the concept of a Tech-Prep program and six-year plan introduced to 
middle and/or high school students? 

Area 7 - Integration of Curriculum and Contextual Teaching and Learning 

14. In what ways has the consortium's activities and services contributed to the 
implementation of integrated curricula and/or contextual teaching methodologies? 

15. Can the consortium document implementation of integrated curricula and/or contextual 
teaching methodologies in secondary and/or post-secondary classrooms? Which ones? 

1 6. Can the consortium document success of the implementation of integrated 

curricula and/or contextual teaching methodologies in secondary and/or post-secondary 
classrooms? How is success documented? 
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Area 8 - Involvement of Academic Departments 

1 7. What is the extent of the involvement of secondary and/or post-secondary teachers from 
academic areas (e.g., mathematics, science, and communications), in the development 
and implementation of Tech-Prep six-year plans (programs of study)? 

1 8. What documentation do you have to support this involvement? 

Area 9 - Exemplary Tech-Prep Activities 

1 9. What Tech-Prep activities conducted by your consortium do you, your governing board, 
members of your consortium, or other individuals or entities recognize as exemplary? 

Area 10 - Barriers to Implementation of Tech-Prep 

20. What barriers to the effective implementation of Tech-Prep programs and processes in 
your consortium have been identified by you, your governing board, members of your 
consortium, or other individuals or entities? 

2 1 . What strategy and/or strategies are you utilizing to address these barriers? 

22. What external support or assistance do you need to overcome these barriers? 

Area 11- Coordination with High Schools That Work fHSTW) 

23. How are High Schools That Work (HSTW) sites coordinating their efforts to include 
Tech-Prep programs and activities? 

24. Do you have a process to identify and disseminate key practices of these dual (Tech-Prep 
and HSTW) sites? 

a) If so, do you have plans for dissemination of these effective practices within your 
consortium? 

Area 12- Use and Application of Technology in Tech-Prep 

25. Can you document secondary and/or post-secondary professional development sessions 
and attendance that focus on the use and application of technology in the classroom in 
support of Tech-Prep programs? 
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Area 13 - Work-Based Learning Activities 



26. List the types of work-based learning activities that are available for Tech-Prep students 
in your consortium? 

a) Secondary - 

b) Post-secondary - 

27. How many secondary Tech-Prep students have participated (and/or currently 
participating) in these activities? 

a) Secondary - 

b) Post-secondary - 

Area 13 - Preparatory Services 

28. What types of preparatory services do members of your consortium provide that assist 
students specifically to prepare for successful participation in Tech-Prep programs? 

Area 14 - Graduate Follow-Up 

29. What types of student follow-up procedures, if any, are being utilized within the 
consortium to help determine appropriate placement and employment of Tech-Prep 
graduates? 

Area 15 - Special Populations 

30. What procedures are being implemented by consortium members that ensure equal access 
to Tech-Prep programs to individuals that are members of special populations? 

31. Are any services being implemented appropriate to the needs of special populations that 
are specific to Tech-Prep programs? 

32. Have any Tech-Prep programs been developed in your consortium that specifically serve 
individuals that are members of special populations? 
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A Participatory, Peer-Review Process 



Introduction 



The purpose of the site evaluation process is to collect information and document the 
characteristics and operations of Tech-Prep consortia to determine if they are satisfactorily 
meeting the requirements of Tech-Prep education as detailed within the consortium's state- 
approved five-year plan. The site visit process will aid in the identification of the strengths and 
barriers in a consortium's Tech-Prep initiative. As a result, areas of strength can be expanded 
while barriers can be addressed and corrected, allowing the consortium, and the initiative's 
effectiveness, to be in a continuous state of improvement. In addition, the peer-review process 
provides an opportunity for the sharing of ideas and experiences and the dissemination of best 
practices statewide. 

Review Cycle 

The Tech-Prep consortium site evaluation process is designed to complete of cycle in three years 
beginning in PY 2001. Up to eight consortia will be visited each year and the order of consortia 
visited determined by a drawing at the beginning of the cycle. Visits will be conducted August 
through June [Insert link here to the file called Site Visit Schedulel . 

Purpose of Site Visits 

The objectives of Tech-Prep consortium site visits are: 

• To document the extent to which each consortium is attaining its goals/objectives as stated in 
its annual funding proposal to the Texas Higher Education Coordinating Board and 5-year 
strategic plan. 

• To observe and document positive examples of leadership that facilitate secondary and post- 
secondary Tech-Prep activities within the consortium, including, those components stated in 
state and federal Tech-Prep legislation. 

• To collect information on the characteristics of each Tech-Prep consortium, including the 
consortium's membership and leadership structure. 

• To collect data to determine the impact of Tech-Prep activities on member institutions and 
their students by identifying measurable benchmarks and parameters and, where possible, 
identifying and utilizing existing data sources that can be used for future comparison. 

• To collect information to determine the State of Texas' role in supporting its local Tech-Prep 
consortia and to provide technical assistance. 

• To recommend actions for improvement of the consortium's Tech-Prep activities. 
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Site Visit Structure 

A team of reviewers selected to offer a broad range of knowledge and experiences will conduct 
each site visit. The team, composed of up to eight people, will be selected to include secondary 
and post-secondary educators from a pool of individuals nominated by Tech-Prep consortia. 

Pairs of reviewers, as appropriate, will conduct each programmatic section of the review. A state 
staff member will lead the site visit team and will coordinate the overall site visit process. Staff 
of the Texas Higher Education Coordinating Board and the Texas Education Agency will 
oversee the site visit process. 

Composition of Team Members 



Each site visit team will be composed of the following, as possible and appropriate: 

1. A secondary teacher, counselor, or administrator who is familiar with Tech-Prep 
programs, activities, and processes; 

2. A post-secondary teacher, counselor, or administrator who is familiar with Tech- 
Prep programs, activities, and processes; 

3. The director/coordinator from the Tech-Prep consortium whose visit immediately 
proceeded this visit; 

4. The director/coordinator from the Tech-Prep consortium whose visit immediately 
follows this visit; 

5. Two members of the staff of the Texas Higher Education Coordinating Board, 
one programmatic and one fiscal; 

6. One member of the staff of the Texas Education Agency; and 

7. The director of the state's Tech-Prep evaluation project. 

Preparation for Site Visits 

Each consortium to be visited will be contacted in advance by state staff in order to assist in the 
coordination and development of the schedule and to identify and contact key personnel in the 
consortium that need to be involved in the visit. Staff of the Tech-Prep evaluation project will 
coordinate logistics of housing and travel with team members. 

The director of the Tech-Prep consortium is responsible for completion of the following 
activities prior to the site visit: 
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1. Self-Study - Each director must complete the self-study instrument and return it to 
the Texas Higher Coordinating Board at least 14 calendar days prior to the site 
visit for dissemination to team members. This instrument will help the director 
prepare for the site visit and will provide team members with pertinent 
information regarding the status, achievements, and barriers to Tech-Prep 
activities [Insert link here for the file called Site Visit Self Studvl : 

2. Schedule Interviews - The director should select, and notify in a timely manner, a 
broad range of individuals representing secondary and post-secondary institutions, 
and business and industry, for participation in interviews and meetings (there 
should be an adequate number of representatives at each interview); 

3. Arrange agenda - The director should make room assignments for scheduled 
meetings, entrance/exit interviews, and team meeting room, based on the sample 
agenda; and 

4. Prepare On-Site Review Materials - The director should assemble and make 
available the requested site visit documentation for review by the site visit team. 

Sample Agenda 

The local Tech-Prep director will be responsible for coordinating and scheduling the site visit itinerary. 

Specific times should be set for the interviews and obsen’ations. The site visit team will arrive the 

evening prior to the day of the on-site visit so the visit can begin at 8:00 a.m. The following sample 

agenda is provided as a guide for the types of activities to be scheduled during the site visit. 

Day One (Wednesday! - Organizational and Process Overview Meetings 

3:00 p.m. Team members arrive at the hotel and check in. 

4:00 p.m. Team members meet with the Tech-Prep director and consortium staff for 

an overview of the process. 

6:00 p.m. Team dinner 

7:30 - 9:00 p.m. Team meeting to review schedule of activities 

Day Two (Thursday) 

8:00 a.m - 5:00 p.m. The team will divide into pairs and will conduct interviews and review 
assigned materials. 

8:00 a.m. The local Tech-Prep director will provide an overview of the consortium 

and its function to date and will provide team with location of materials. 

8:30 a.m - 10:00 a.m. Interview # 1 - Selected team members will interview secondary, post- 
secondary, or business and industry representatives. 

1 0:00a. m - 1 1:30 a.m. Interview # 2 - Selected team members will interview secondary, post- 
secondary, or business and industry representatives. 

12:00 noon Team member luncheon (may include consortium representatives) 
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1:30 - 3:00 p.m. 


Interview # 3 - Selected team members will interview secondary, post- 
secondary, or business and industry representatives. 


3:00 - 5:00 p.m. 


Team members complete review of materials and interview consortium 
staff, as needed. 


5:00 - 5:30 p.m. 


Dav One Exit Conference - The site visits team will meet with the local 
Tech-Prep Coordinator to finalize the day two itinerary. 


6:00 p.m. 


Team dinner 


7:30 - 9:00 p.m. 


Site Visit Team Conference - The site visit team will meet privatelv to 
summarize their findings and determine if any issues exist for redress 
during the morning of day two and to prepare for the exit interview. 



Day Three (Friday) 

8:00a.m. - 10:00 a.m. The team will divide into pairs and will continue to review assigned 

materials and get clarification from consortium staff. 

10:00 — 1 1 :00 a.m. Site Visit Team Conference - The site visit team will meet privately to 
summarize their findings to prepare for the exit interview. Strengths and 
noted barriers will be identified, and commendations and 
recommendations will be specified. 

1 1:00a.m. -12:30 p.m. Exit Interview - The team will meet with appropriate consortium 
administration, including but not limited to: school district 
superintendents, principals, community college president and deans, local 
Tech-Prep director, fiscal agents staff, and interested secondary and post- 
secondary participants, to present findings and recommendations. 

Site Visit Follow-up 

Following the site visit, the team leader (state staff) will prepare and provide to the consortium's 
leadership a written report within 30 days summarizing the results of the visit. Specific strengths 
and weaknesses will be identified and recommendations for continuation and dissemination of 
exemplary practices and suggestions to address areas identified as needing improvement. 

The consortium will have the opportunity to respond to the report in writing to the Texas Higher 
Education Coordinating Board. 



Permission to reprint material obtained from Texas State Tech Prep Director, TX. 
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Tech-Prep Consortium Evaluation for : 

Site Visit Dates: Name of Reviewer: 



Review Area: Program Design 



Sub-Area: Articulation 



Description of Criteria: Tech-Prep programs function within an articulation agreement 

developed under the guidance of a discipline specific team that specifies 
courses and course sequences, the responsibilities of the educational 
partners, and the process by which students can receive college credit. 



Resources: Documentation of articulation agreements; Documentation of annual discipline 

specific team meetings; Documentation of state approved Tech-Prep program 6- 
year plans; consortium member documents regarding articulation of Tech-Prep 
credit (eg. catalogs, brochures, bulletins, etc.). 



Consortium Quality 


Reviewer Summary 


Measure: 


Annual meetings are held to develop and 


Meets Standard 




improve programs. 


Does Not Meet Standard 


Core 




Comments: 


Standard: 


Consortium has evidence that opportunity is 
offered for the joint review by secondary and 
postsecondary faculty of Tech-Prep program 
course sequences and curricula. 


Meets Standard 


Sub- Area: 


Such meetings are held on a regular basis. 


Does Not Meet Standard 
Comments: 


Source: 


Document Review 


Measure: 


All Tech-Prep programs have written 


Meets Standard 




articulation agreements. 


Does Not Meet Standard 


Core 




Comments: 


Standard: 


All state approved Tech- Prep programs function 
within current written agreements that specify 
the courses and course sequence leading to a 
specific career or career cluster, the roles and 
responsibilities of the educational partners, and 
the mechanism by which students can receive 
college credit. 




Sub-Area 


Consortium has a record of articulation 


Meets Standard 




agreement updates and a sample agreement 


Does Not Meet Standard 




from each participating college. 


Comments: 


Source: 


Document Review. 
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Tech-Prep Consortium Evaluation for : 

Site Visit Dates: Name of Reviewer: 



Review Area: Program Design 



Sub-Area: Implementation 



Description of Criteria: Tech-Prep programs lead to high-skill, high-wage employment in 

occupations with growth potential. Advisory committees are used in the 
development and maintenance of programs at both the secondary and 
postsecondary levels. Tech-Prep programs, program support, and 
Professional development activities are available to all high schools and 
colleges in the service area. 



Resources: State and national occupational statistics; Local occupational information 

including business surveys; Job descriptions and workplace needs from advisory 
committee members; State and national occupational skill standards; Minutes of 
AAS degree program advisory committee meetings 



Consortium Quality 


Reviewer Summary 


Measure: 


All public secondary districts and/or high 


Meets Standard 




schools are contacted regularly. 


Does Not Meet Standard 


Core 

Standard: 


The consortium can document that every public 
school district in the planning region with a 
high school has been provided information on 
the opportunity to develop Tech- Prep programs 
and to attend consortium- sponsored staff 
development activities. 


Comments: 


Sub- Area: 


The consortium may also provide for additional 
participation opportunities such as mini-grants 
to schools for specific projects. 


Exceeds Standard 
Comments: 


Sources: 


Documents, Secondary interviews. 




Measure: 


Industry competencies are used in the 


Meets Standard 




development Tech-Prep programs. 


Does Not Meet Standard 


Core 

Standard: 


Each Tech-Prep program of study is reviewed 
regularly by the college's business/industry 
AAS degree program advisory board for up-to- 
date industry standards. 


Comments: 


Sub- Area: 


The college advisory boards for Tech-Prep 


Exceeds Standard 




programs include secondary representation. 


Comments: 


Sources: 


Documents; secondary, postsecondary, and 
business/industry interviews. 
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Tech-Prep Consortium Evaluation for : 


Site Visit Dates: 


Name of Reviewer: 


Review Area: Program Design 


Sub-Area: Seamless Course of Study 



Description of Criteria: 



Tech-Prep students are enrolled in a seamless program of study that 
begins at least by grade eleven and continues through at least two years 
of postsecondary education incorporating broad career clusters and 
rigorous academic and technical competencies. 



O 

ERIC 



Resources: Tech-Prep six-year educational plans and articulation agreements; Interviews 

with counselors, registrars, and students; College registration policies regarding 
awarding of credit for articulated courses; List of programs in place. 



Consortium Quality 


Reviewer Summary 


Measure: 


Six-year coherent sequence of courses leading 


Meets Standard 




to a two-year postsecondary credential. 


Does Not Meet Standard 


Core 

Standard: 


Each Tech-Prep education plan is current, 
relevant, and meet recommended high school 
graduation program requirements, and leads to 
at least a two-year degree, certificate, or 
apprenticeship. Programs follow a logical 
sequence of rigorous coursework that allows 
students to build on acquired knowledge 
without repetition. 


Comments: 


Sub- Area: 


TP programs are articulated with baccalaureate 


Exceeds Standard 




programs. 


Comments: 


Source: 


Document review. 




Measure: 


Students are provided a system to easily 
transition to postsecondary education. 


Strong Evidence of Progress 
Some Evidence of Progress 


Core 




No Evidence of Progress 


Standard: 


Systems are in place to ensure that students 
receive program information, counseling, and 
college credit (as applicable) with minimum 
difficulty. Programs offer monetary savings 
and/or are time- shortened for students who 
have completed the high school portion of the 
program. 


Comments: 


Sub-Area: 


Consortium staff are knowledgeable about each 


Meets Standard 




college system for identifying TP students and 


Does Not Meet Standard 




awarding credit. 


Comments: 


Source: 


Document review, secondary and 
postsecondary interviews. 
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Tech-Prep Consortium Evaluation for : 

Site Visit Dates: Name of Reviewer: 



Review Area: Program 



Sub-Area: Program Support 



Description of Criteria: Consortium members should be aware of the differences in student 

backgrounds that affect their ability to complete a Tech-Prep program. 
A consortium should help provide career awareness and career 
exploration activities to encourage students to explore non-traditional 
careers that their gender or ethnic background would otherwise 
discourage them from attempting. 



Resources: Documentation of professional development activities and attendance records, 

informational materials produced by the consortium, survey responses from 
students indicating that they have participated in career awareness activities and 
have been counseled regarding non-traditional career pathways. 



Consortium Quality 


Reviewer Summary 


Measure: 


The consortium addresses the ethnic, cultural. 


Meets Standard 




and economic barriers that inhibit program 
participation. 


Does Not Meet Standard 
Comments: 


Exemp. 






Standard: 


Consortium staff can identify special 
populations services provided by schools and 
colleges offering Tech-Prep programs in the 
region. Example: Spanish language 
informational and recruiting material. 




Sub- Area: 


TP programs are articulated with baccalaureate 
programs 




Sources: 


Document review, consortium staff interviews. 




Measure: 


Tech-Prep programs have been developed, and 


Meets Standard 




are being used, that lead to non-traditional 
careers. 


Does Not Meet Standard 
Comments: 


Exemp. 






Standard: 


Consortium staff can identify specific programs 
in the service region that lead to non-traditional 
careers. 




Sub- Area: 


Consortia exceed the standard by documenting 


Exceeds Standard 




the effectiveness of the program(s) by under- 
represented student participation. 


Comments: 


Sources: 


Document review, consortium staff interviews. 
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Tech-Prep Consortium Evaluation for : 
Site Visit Dates: 



Name of Reviewer: 



Review Area: Instruction 



Sub-Area: Integrated and Contextual Learning 



Description of Criteria: 



Tech-Prep program content is taught by integrating academic and technical 
content to reinforce student learning. Classes utilize realistic applications of 
knowledge and skills to demonstrate the utility of that knowledge and/or skill 
within the context of the workplace. 



Resources: Documentation of professional development activities; presence of an active and 

successful High Schools That Work site; student survey responses, student completion of, 
and success in, program courses; documentation of use of specific curricular materials and 
activities such as Principles of Technology or Applied Algebra. 



Consortium Quality 


Reviewer Summary 


Measure : Consortium staff have provided professional 

development opportunities and materials to 
member schools. 

Core 

Standard: Consortium staff can document that materials 
and professional development activities 
emphasize the importance of curriculum 
integration and contextual teaching strategies in 
Tech- Prep programs. 

Sources: Document review, secondary and postsecondary 

interviews. 


Secondary . t 

Strong Evidence of Progress 
Some Evidence of Progress 
No Evidence of Progress 
Comments: 

Postsecondary 

Strong Evidence of Progress 
Some Evidence of Progress 
No Evidence of Progress 
Comments: 


Measure: Member schools are utilizing integrated and 


Secondary 


contextual teaching and evaluation strategies. 


«/ 

Strong Evidence of Progress 


Exemp. 


Some Evidence of Progress 


Standard: Consortium can document schools and/or 


No Evidence of Progress 


teachers that utilize integration and contextual 


Comments: 


methodologies. 






Postsecondary 




Strong Evidence of Progress 




Some Evidence of Progress 




No Evidence of Progress 




Comments: 


Sub- Area Documentation also shows the use of 




alternative assessment techniques and 
competency-based instruction. 


Strong Evidence of Progress 




Some Evidence of Progress 


Sources: Document review, secondary and postsecondary 


No Evidence of Progress 


interviews. 


Comments: 
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Tech-Prep Consortium Evaluation for : 



Site Visit Dates: Name of Reviewer: 



Review Area: Instruction 



Sub-Area: School-site/Work-site Learning 



Description of Criteria: Tech-Prep programs should include, whenever possible, work-based 

learning experiences that allow students to understand all aspects of the 
industry and how their school-site instruction is utilized in business and 
industry. Experiences may include, but are not limited to, shadowing 
experiences, internships, cooperative education or school-site programs 
such as Project REAL. 



Resources: Documentation of program sequences that identify work-based learning components, 
materials and participation data for various consortium and school-sponsored 
activities, documentation of schools providing external learning experiences, data 
indicating the extent of employer participation in such programs. 



Consortium Quality 



Reviewer Summary 



Measure: Consortium members offer regional students the 

opportunity to participate in work-based 
learning activities. 

Exemp. 

Standard: Consortium can provide a list of work-based 
activities offered students. Every Tech-Prep 
program should contain at least one such 
experience. 



Secondary 

Strong Evidence of Progress 

Some Evidence of Progress 

No Evidence of Progress 

Comments: 



Source: 



Measure: 



Exemp. 

Standard: 



Source: 



Document review; secondary, postsecondary, 
and business/industry interviews. 



Consortium can document member schools and 
students participating in work-based learning 
experiences. 



Consortium can provide a list of schools and 
student participation data for work-based 
experiences. Consortium may provide 
documentation indicating business and industry 
participation in work-based learning 
experiences. 



Document review. 



Postsecondary 

Strong Evidence of Progress 

Some Evidence of Progress 

No Evidence of Progress 

Comments: 



Secondary 

Exceeds Standard 

Meets Standard 

Does Not Meet Standard 

Comments: 

Postsecondary 

Exceeds Standard 

Meets Standard 

Does Not Meet Standard 

Comments: 
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Tech-Prep Consortium Evaluation for : 


Site Visit Dates: 


Name of Reviewer: 


Review Area: Instruction 


Sub- Area: Use of Educational Technology 



Description of Criteria: 



Consortia should encourage and provide training for the use of technology in 
the classroom in the delivery of instruction and information about Tech-Prep 
programs. Technology includes, but is not limited to, distance education, 
computer-assisted instruction, Internet Web sites, electronic meetings, and 
computer simulation software. 

Resources: Documentation of professional development workshops, attendance data, consortium 

and/or school Web sites, evidence of technology purchases, student survey responses 
indicating the extent of the use of technology in the classroom, documentation of Tech- 
Prep courses taught through distance learning. 



Consortium Quality 


Reviewer Summary 


Measure : Technology is used in the deliver}' of Tech-Prep 


Strong Evidence of Progress 


courses. 


Some Evidence of Progress 


Exemp. 


No Evidence of Progress 
Comments: 


Standard: Regional teachers have had the opportunity to 




attend professional development workshops on 




the use of technology in the classroom and the 




consortium has assisted in the acquisition of 




technology where possible. 




Sub- Area: Tech- Prep courses may be taught via distance 


Exceeds Standard 


learning or through computer-assisted 


Comments: 


instruction. 




Source: Document review; secondary and postsecondary 




interviews. 




Measure: Technology is utilized in the communication of 


Exceeds Standard 


Tech-Prep programs and activities. 


Meets Standard 


Exemp. 


Does Not Meet Standard 


Standard: The consortium has a current web site, with 


Comments: 


links to other relevant sites, to provide 




information on Tech-Prep programs, 




articulation procedures, and consortium 




activities. 




Sub- Area.* Consortium staff utilizes technology in the 


Strong Evidence of Progress 


deliver}' of information to various audiences 


Some Evidence of Progress 


including professional development. 
Consortium may use technology to facilitate 


No Evidence of Progress 


electronic meetings and communicate with 


Comments: 


members to exceed standards. 




Source: Document review; staff, secondary, and 




postsecondary interviews. 
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Tech-Prep Consortium Evaluation for : 

Site Visit Dates: Name of Reviewer: 



Review Area: Counseling 



Sub-Area: Career Exploration 



Description of Criteria: Beginning at least by the eighth grade, students should have the 

opportunity to participate in career exploration activities that expose 
them to the day-to-day activities of the workplace and allow them to 
experience different career and non-traditional occupations. 
Experiences may be either work-site related or gained by alternative 
methods such as on-line or computer-based. 



Resources: Documentation of school and student participation in local and consortium-wide 

activities, documentation of schools that have career exploration activities, 
documentation of business/ industry mentors and speakers, documentation of the 
results of such activities as Groundhog Day, Career Day, and career-related 
mentoring. 



Consortium Quality 


Reviewer Summary 


Measure: 


The consortium and its member secondary 
schools provide opportunities for career 


Exceeds Standard 




exploration. 


Meets Standard 






Does Not Meet Standard 


Core 

Standard: 


The consortium provides opportunities for 
career exploration to students and there is 
evidence of an increasing number of member 
schools offering local career exploration 
activities, beginning by the eighth grade, to 
their students. 


Comments: 


Sources: 


Document review, secondary interviews. 




Measure: 


Students receive career guidance in the setting 
of career and educational goals. 


Exceeds Standard 
Meets Standard 


Core 




Does Not Meet Standard 


Standard: 


There is evidence that secondary students 
receive information on Tech-Prep six-year plans 
and related career opportunities. 


Comments: 


Sources: 


Document review, secondary interviews. 
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Tech-Prep Consortium Evaluation for : 


Site Visit Dates: 


Name of Reviewer: 


Review Area: Professional Development 


Sub-Area: All Aspects of the Industry 



Description of Criteria: 



Secondary and postsecondary faculty and counselors involved in Tech-Prep 
programs must have exposure to the workplace. Faculty need to know how 
their particular discipline applies to various business and industry application 
to help students relate theory to application. Counselors need to understand 
workplace opportunities and needs. 



Resources: Documentation of professional development opportunities and attendance records, 

documentation of business and industry involvement, products of worksite experiences 
including curriculum plans and classroom activities. 



Consortium Quality 


Reviewer Summary 


Measure: 

Exemp. 

Standard: 


Educators participate in worksite professional 
development activities. 

Data shows an increase in the numbers of 
educators that participate in work-site 
professional development activities such as 
shadowing experiences or internships. 


Secondary 

Strong Evidence of Progress 
Some Evidence of Progress 
No Evidence of Progress 
Comments: 


- 






Postsecondary 

Strong Evidence of Progress 
Some Evidence of Progress 
No Evidence of Progress 
Comments: 




Sub- Area: 


Materials are produced as a result of these 
activities that can be used in the classroom. 


Exceeds Standard 
Comments: 




Sources: 


Document review, secondary and post secondary 
interviews. 






Measure: 

Exemp. 

Standard: 


Businesses and industries provide sites and 
experiences for educator professional 
development activities. 

Data shows an increase in the number of 
businesses and sites that are available for 
educator professional development activities 
such as shadowing experiences and internships. 


Strong Evidence of Progress 
Some Evidence of Progress 
No Evidence of Progress 
Comments: 




Sources: 


Document review, business/industry, 
secondary, and postsecondary interviews. 
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Tech-Prep Consortium Evaluation for : 



Site Visit Dates: Name of Reviewer: 



Review Area: Professional Development Sub-Area: Assessment and Staff Knowledge 

Description of Criteria: Secondary and postsecondary faculty must be provided professional 

development activities that offer alternatives to traditional student 
assessment techniques. Secondary and postsecondary faculty must be 
provided professional development activities that address the use of 
educational technologies. 



Resources: Documentation regarding professional development activities including attendance 

records, student survey responses regarding the use of technology in the 
classroom, examples of alternative assessment techniques used in member schools. 



Consortium Quality 


Reviewer Summary 


Measure: 


Opportunities have been made available for 
faculty to learn alternative assessment 


Meets Standard 




techniques. 


Does Not Meet Standard 






Comments: 


Exemp. 

Standard: 


Data indicates that professional development 
activities have been made available to 






secondary and post secondary faculty in 
alternative assessment including portfolios 
and competency-based evaluation. 




Sub- Area: 


Data may indicate an increase in the numbers 


Exceeds Standard 




of postsecondary faculty taking advantage of 
the professional development. 


Comments: 


Sources: 


Document review, secondary and 
post secondary interviews. 




Measure: 


Staff have acquired knowledge and 
administrative skills regarding the entire 


Exceeds Standard 




Tech-Prep program process. 


Meets Standard 
Does Not Meet Standard 


Exemp. 

Standard: 


All consortium staff members have been 


Comments: 




given the opportunity to acquire necessary 
knowledge and skills (may be local training) 
regarding Tech- Prep programs, articulation 
processes, and other relevant knowledge. 




Sources: 


Document review, consortium staff 
interviews. 
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Tech-Prep Consortium Evaluation for : 
Site Visit Dates: 



Name of Reviewer: 



Review Area: Professional Development 



Sub-Area: Career Guidance/Student Identification 



Description of Criteria: Secondary and postsecondary counselors must be given the opportunity 

to participate in professional development that will provide the tools 
necessary to deliver career guidance and counseling to students. 
Secondary and postsecondary personnel must be trained in the correct 
identification of Tech-Prep students, Tech-Prep programs, and 
articulation processes. 

Resources: Secondary and postsecondary student participation data, workshop documentation 

including attendance, credit articulation records, student completion data, 
workshop materials, student survey responses to questions about career guidance 
activities. 



Consortium Quality 


Reviewer Summary 


Measure: Counselors are given the opportunity to 


n i 


participate in career guidance training 


Secondary 


activities. 


Exceeds Standard 




Meets Standard 


Core 


Does Not Meet Standard 


Standard : Consortium has provided the opportunity for 


Comments: 


counselors to receive career guidance 




training. The consortium may show an 




increase in the numbers of participants 
including postsecondary counselors. The 


Postsecondary 


consortium may show an increase in career 


Exceeds Standard 


counseling activities in both secondary and 


Meets Standard 


postsecondary institutions. 


Does Not Meet Standard 


Sources! Document review. 


Comments: 


Measure : The consortium has provided training in the 


Meets Standard 


correct identification of Tech-Prep students. 


Does Not Meet Standard 


Core 


Comments: 


Standard: Documentation, including workshop 




materials and attendance records, shows that 




the consortium has participated in the 




delivery of professional development related 




to the correct identification of Tech-Prep 




students including PEIMS and college entry 




and articulation processes to both secondary 




and postsecondary audiences. 




Sources: Document review, secondary and 




postsecondary interviews. 
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Tech-Prep Consortium Evaluation for : 



Site Visit Dates: Name of Reviewer: 



Review Area: Marketing 



Sub-Area: Dissemination to Target Audiences 



Description of Criteria: The consortium has developed information dissemination plans, 

marketing strategies, and related print and electronic materials, 
including a currently updated Web site, that are disseminated 
appropriately to target audiences. 



Resources: Consortium Web site, marketing materials, marketing plan, documentation of 

information distribution, records of Web site hits, examples of marketing materials 
and dissemination plans for targeted audiences. 



Consortium Quality 


Reviewer Summary 


Measure: 


Students have information on TP programs. 


Secondary 

Meets Standard 




articulated credit, and workforce/career data. 


Core 




Does Not Meet Standard 


Standard: 


Informational materials are provided to 
secondary and postsecondary students on the 
benefits of Tech- Prep, postsecondary 
programs offered, career opportunities, and 
the process required to receive articulated 


Comments: 




credit. 


Postsecondary 

Meets Standard 
Does Not Meet Standard 
Comments: 


Sources: 


Document review, secondary and 
postsecondary interviews. 




Measure: 


Employers, parents, and communities have 


Meets Standard 




information on TP programs, articulated 


Core 


credit, and workforce/career data. 


Does Not Meet Standard 
Comments: 


Standard: 


Employers, parents, and local communities 
are provided information and informational 
materials on the benefits of Tech- Prep, 
postsecondary programs offered, career 
opportunities, and the process required to 
receive articulated credit. 




Sources: 


Document review, business/industry. 
Interviews 
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Tech-Prep Consortium Evaluation for : 



Site Visit Dates: 



Name of Reviewer: 



Review Area: Marketing 



Sub-Area: Dissemination to Target Audiences 



Description of Criteria: The consortium has developed information dissemination plans, 

marketing strategies, and related print and electronic materials, 
including a currently updated Web site that are disseminated 
appropriately to target audiences. 



Resources: Consortium Web site, marketing materials, marketing plan, documentation of 

information distribution, records of Web site hits, examples of marketing 
materials and dissemination plans for targeted audiences. 



Consortium Quality 



Reviewer Summary 



Measure: Educators have information on TP programs, 

articulated credit, and workforce/ career data. 



Core 

Standard: 




Secondary and postsecondary educators are 
provided information on the benefits of Tech- 
Prep, postsecondary programs offered, career 
opportunities, and the process required to 
receive articulated credit. 



Secondary 

Meets Standard 

Does Not Meet Standard 

Comments: 



Sources: 



Measure: 



Core 

Standard: 



Sources: 




Postsecondary 

Meets Standard 

Does Not Meet Standard 

Comments: 



Document review, secondary and postsecondary 
interviews. 



Counselors have information on TP programs, 
articulated credit, and workforce/career data. 



Secondary and postsecondary counselors are 
provided information on the benefits of Tech- 
Prep, postsecondary programs offered, career 
opportunities, and the process required to 
receive articulated credit. 



Secondary 

Meets Standard 

Does Not Meet Standard 

Comments: 



Document review, secondary and postsecondary 
interviews. 



Postsecondary 

Meets Standard 

Does Not Meet Standard 

Comments: 
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Tech-Prep Consortium Evaluation for : 



Site Visit Dates: Name of Reviewer: 



Review Area: Budgetary 



Sub-Area: Record-keeping 



Description of Criteria: Every consortium, as part of the evaluation process, will receive fiscal 

monitoring by an Agency accountant. This form is a summary of the 
fiscal monitoring that ensures that all consortium expenditures are 
appropriately documented and are provided for and detailed in the grant 
budget for the current year. 



Resources: Fiscal agent records and consortium fiscal records, supporting documentation as 

appropriate. 



Consortium Quality 


Reviewer Summary 


Measure: 


The consortium and/or its fiscal agent keep 
accurate and complete records on all 
expenditures. 


Meets Standard 
Does Not Meet Standard 


Core 

Standard: 


All expenditures have supporting 
documentation. 


Comments: 


Sources: 


Document review. 




Measure: 


Consortium maintains records on how funds are 
distributed. 


Meets Standard 
Does Not Meet Standard 


Core 

Standard: 


The consortium has documentation that details 


Comments: 




how funds are budgeted with governing board 
and member input. 




Sub Area: 


If the consortium provides mini -grants to 
member schools, there is documentation that 
describes the award process. 


Meets Standard 
Does Not Meet Standard 




Comments: 


Sources: 


Document review. 
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Tech-Prep Consortium Evaluation for : 



Site Visit Dates: Name of Reviewer: 



Review Area: Budgetary 



Sub-Area: Budgetary Monitoring 



Description of Criteria: Every consortium, as part of the evaluation process, will receive fiscal 

monitoring by an Agency accountant. This form is a summary of the 
fiscal monitoring that ensures that all consortium expenditures and 
mini-grant expenditures are in accordance with federal and state 
guidelines. 



Resources: Fiscal agent records and consortium fiscal records, supporting documentation as 

appropriate. 



Consortium Quality 


Reviewer Summary 


Measure: 


All consortium expenditures are within Perkins 


Meets Standard 
Does Not Meet Standard 




Act. state guidelines, and EDGAR guidelines. 


Core 




Comments: 


Standard: 


Tech-Prep funds are expended according to 
Perkins Act guidelines as well as EDGAR 
regulations regarding all expenditures of federal 
funds. Additionally, funds are expended 
according to specific state guidelines including, 
but not limited to, the requirements set forth in 
the RFP “Provisions and Assurances”. 




Sources: 


Document review. 




Measure: 


All mini-grant expenditures are within Perkins 
Act. state guidelines, and EDGAR guidelines. 


Meets Standard 
Does Not Meet Standard 


Core 




Comments: 


Standard: 


The consortium has documentation that mini- 
grant funds are applied for, documented, and 
expended in a manner that is consistent with 
Perkins Act, EDGAR regulations, and state 
guidelines. 




Sub Area: 


Mini-grant recipients must be made aware of 


Meets Standard 




the restrictions placed on Perkins Title II funds 


Does Not Meet Standard 




as part of the RFP process. 


Comments: 


Sources: 


Document review. 
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Tech-Prep Consortium Evaluation for : 


Site Visit Dates: 


Name of Reviewer: 


Review Area: Planning 


Sub-Area: Strategic Planning 


Description of Criteria: 


The consortium must have a 5-year plan including: implementation and 
dissemination of Tech-Prep programs of study; curriculum 
development; student transition to college and work; professional 
development in counseling, real-world applications; academic/technical 
course integration; marketing; and integration of the workplace into the 



educational process. 



Resources: Consortium 5-year plan, documentation indicating, at a minimum, annual 

updating of the plan including participation records, documentation indicating 
evaluation procedures utilized and the results of the evaluations. 



Consortium Quality 



Reviewer Summary 



Measure: The consortium has a long-range 

implementation plan. 

Core 

Standard: The consortium has on file a current, 

5-year implementation plan that incorporates 
state and local goals. The plan includes 
objectives, activities, dates, and evaluation 
criteria. 



Meets Standard 

Does Not Meet Standard 

Comments: 



Sources: 



Measure: 



Core 

Standard: 



Document review. 



The consortium plan is re-visited and updated 
on a regular basis. 



Meets Standard 

Does Not Meet Standard 

Comments: 



The 5-year plan is reviewed, evaluated, and 
revised (as needed) by the governing board and 
fiscal agent at least annually. Documentation 
shows that the long-range plan has been 
revisited and revised as needed. 



Sub Area: There is a process that affords all consortium 
members the opportunity to offer input into the 
consortium plan. 



Meets Standard 

Does Not Meet Standard 

Comments: 



Sources: Document review. 
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Tech-Prep Consortium Evaluation for : 




Site Visit Dates: 


Name of Reviewer: 


Review Area: Planning 


Sub-Area: Local Participation in Planning 



Description of Criteria: 



All consortium member institutions must have the opportunity to 
participate in the planning of consortium activities. In the event that it 
is impossible for every educational entity to participate, they must have 
representation and a venue for input into the process. 



Resources: Consortium by-laws, documentation of meeting attendance for planning and 

budget development/modification, documentation regarding the dissemination of 
consortium goals, objectives, activities, and budgetary information. 



Consortium Quality 


Reviewer Summary 


Measure: 


The structure and governance of the consortium 


Meets Standard 




provides for input opportunities for all 




stakeholders. 


Does Not Meet Standard 
Comments: 


Core 






Standard: 


By-laws provide for representation of all 
stakeholders, meetings are adequately 
advertised. 




Sub- Area: 


There is a process where all stakeholders are 


Meets Standard 




given the opportunity to participate in the 


Does Not Meet Standard 




decision-making process. 


Comments: 


Sources: 


Document review. 




Measure: 


Consortium members participate in the 


Meets Standard 




establishment of the consortium's annual budget 




and budget revisions. 


Does Not Meet Standard 
Comments: 


Core 






Standard: 


The consortium can demonstrate that the use of 
a representative budget committee in the 
development of its annual budget and budget 
revisions. 




Sub Area: 


Consortium stakeholders are adequately 


Meets Standard 




notified and provision is made for member in- 


Does Not Meet Standard 




put into the process. 


Comments: 


Sources: 


Document review; business/industry, 
secondary, and postsecondary interviews. 
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Tech-Prep Consortium Evaluation for : 




Site Visit Dates: 


Name of Reviewer: 


Review Area: Student Success 


Sub-Area: Student Participation and Completion 



Description of Criteria: Consortia must show an improvement trend in all Title II Tech-Prep quality 

indicators including completion of the secondary program, matriculation to 
postsecondary education, hours of articulated credit awarded, completion of 
the postsecondary program, and job placement. Additional indicators such as 
drop-out reduction and satisfaction data should be also considered. 



Resources: State data including: Texas Education Agency PEIMS data; Texas Higher Education 

Coordinating Board data; Texas Workforce Commission data (SOICC data); and local 
survey instrument data 



Consortium Quality 


Reviewer Summary 


Measure: Students are successful in secondary Tech-Prep 


Meets Standard 


programs. 


Does Not Meet Standard 


Core 


Comments: 


Standard: Student data indicates an increase in the number 




of Tech-Prep program participants and 




completers at the secondary level. 




Sub- Area: Students indicate satisfaction in the Tech- Prep 


Exceeds Standard 


programs offered at their schools. 


Comments: 


Sub- Area: Data indicates an increase in the number of 


Exceeds Standard 


students participating in work-based 


Comments: 


educational activities. 




n L a _ 


Exceeds Standard 


oUO- Area: Data indicates an increase in the use of 




contextual teaching methodologies. 


Comments: 


Sources: Document review. 




Measure: Students are successful in postsecondary Tech- 


Meets Standard 


Prep activities. 


Does Not Meet Standard 


Core 


Comments: 


Standard: Data indicates an increase in the number of 




students transitioning to postsecondary 




programs and receipt of articulated credit. 




Sub Area: Data indicates student completion of 


Exceeds Standard 


postsecondary programs, and placement in 


Comments: 


appropriate jobs. 




Sources: Document review. 
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Tech-Prep Consortium Evaluation for : 

Site Visit Dates: Name of Reviewer: 



Review Area: Evaluation 



Sub-Area: Benchmarks and Enrollment Data 



Description of Criteria: The consortium has established benchmarks and quality standards 

that can be clearly communicated to consortium members. The 
consortium can provide local program enrollment data. 



Resources: Consortium long-range plan that includes evaluation criteria for state and local 

goals, documentation of local data collection such as Scantron reporting forms 
and local survey data. 



Consortium Quality 


Reviewer Summary 


Measure: Consortium has clearly defined benchmarks and 

evaluation criteria. 

Core 

Standard: Consortium long-range plan has clearly defined 
and established quantitative evaluation criteria 
for each state and local goal and/or objective. 
There may be additional unique criteria 
depending on the consortium’s local goals and 
objectives. 


Meets Standard 
Does Not Meet Standard 
Comments: 


Sources: Document review. 




Measure: Consortium maintains records on approved 

programs and activity participation. 

Core 

Standard: The consortium has current data on all approved 
AAS Tech- Prep programs offered by secondary 
school/school district and college participating. 


Meets Standard 
Does Not Meet Standard 
Comments: 


Sources: Document review. 
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Tech-Prep Consortium Evaluation for : 



Site Visit Dates: Name of Reviewer: 



Review Area: Evaluation 



Sub-Area: Student Follow-up 



Description of Criteria: The consortium has a method for the identification and follow-up of 

Tech-Prep students in both secondary and postsecondary programs. 



Resources: TEA PETMS data, THECB college data, Scantron data, college procedures for 

awarding articulated credit, college procedures for identifying Tech-Prep 
students. 



Consortium Quality 



Measure: Consortium members can identify secondary 

Tech-Prep students. 



Core 

Standard: Consortium member secondary schools have 
method to identify a student who is 
participating in or has completed the high 
school portion of the Tech-Prep program. 



Reviewer Summary 



Strong Evidence of Progress 

Some Progress 

No Evidence of Progress 

Comments: 



Sources: Document review; secondary interview. 



Measure: Consortium members can identify 

postsecondary Tech-Prep students. 

Core 

Standard: Consortium member colleges have a method to 
identify a student who is matriculating from a 
secondary Tech-Prep program and to record the 
award of college credit for classes taken in high 
school, where applicable. 



Strong Evidence of Progress 

Some Progress 

No Evidence of Progress 

Comments: 



Sub-Area: There is evidence of an increasing number of 
students who matriculate to postsecondary 
programs, receive articulated college credit, and 
enter and complete Tech-Prep programs. 



Sources: Document review; postsecondary interview. 



Strong Evidence of Progress 

Some Progress 

No Evidence of Progress 

Comments: 
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APPENDIX H 

Assessment of Tech Prep Components (MT) 
Central Montana Tech Prep Consortium 



Submitted by 



Name of School or College 



Please rate CMTPC's progress on each of the following factors to evaluate the collaborative efforts of the 
CMTPC. Indicate your response by circling the most appropriate number, according to the scale below. 
Please be sure to complete both sides of the form. 



i. 



ii. 



m. 



IV. 



v. 



VI. 



vn. 



0 - N/A or Not enough information to answer 

1 - Nonexistent 

2 - Beginning 



3 - Moderate 

4 - Advanced 

5 - Highly advanced 



VISIBLE COMMITMENT OF LEADERS TO TECH PREP 

Local Board of Trustees/School Board Members 0 1 2 

School District Administrators (Superintendents & Principals) 0 12 

MSUCOT-GF Administration (Deans & Registrar) & Faculty 0 1 2 

Other Colleges' Administration & Faculty 0 1 2 

Community and Business Leaders 0 12 

Central Montana Tech Prep Consortium Staff 0 1 2 



FACULTY AND STAFF ORIENTATION ABOUT TECH PREP 

Local High School & District: Knowledge and awareness 0 1 2 

Local High School & District: Support and active involvement 0 1 2 

MSUCOT-GF: Knowledge and awareness 0 1 2 

Other Colleges/P-S Institutions. Knowledge and awareness 0 1 2 

Other Colleges/P-S Institutions. Support & active involvement 0 1 2 



TECH PREP ARTICULATION AGREEMENTS 

Covering courses in Business & Office Technology 

Covering courses in Allied Health (Chemistry, Biology) 

Covering courses in Trades & Technology 

Covering courses in Family /Consumer Science 

Covering courses in Graphic/Commercial Art 

Covering courses in academic/general education (Math, English) 

Covering courses in other vocational cluster areas 



0 1 2 
0 1 2 
0 1 2 
0 1 2 
0 1 2 
0 1 2 
0 1 2 



INSERVICE & PROFESSIONAL DEVELOPMENT FOR TEACHERS AND COUNSELORS 

Provided by Central Montana Tech Prep Consortium 0 1 2 

Provided by Local High School & District 0 1 2 

CAREER GUIDANCE RELATED TO TECH PREP 

Emphasized by Central Montana Tech Prep Consortium 0 1 2 

Emphasized by Local High School and/or School District 0 1 2 



RELEVANCY OF INSTRUCTION TO LABOR MARKET 

As provided in High School and/or School District 0 1 2 

As provided at MSU College of Technology-Great Falls 0 1 2 

As provided at other Tech Prep partner colleges 0 1 2 



CURRICULUM REVIEW BY LOCAL EMPLOYERS/WORKERS 

Local School District: Use of Business Advisory Panels 0 1 2 

MSUCOT-GF: Use of Business Advisory Panels 0 1 2 



3 4 
3 4 
3 4 
3 4 
3 4 
3 4 



3 4 
3 4 
3 4 
3 4 
3 4 



3 4 
3 4 
3 4 
3 4 
3 4 
3 4 
3 4 



3 4 
3 4 



3 4 
3 4 



3 4 
3 4 
3 4 



3 4 
3 4 



5 

5 

5 

5 

5 

5 



5 

5 

5 

5 

5 



5 

5 

5 

5 

5 

5 

5 



5 

5 



5 

5 



5 

5 

5 



5 

5 
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0 - N/A or Not enough information to answer 

1 - Nonexistent 

2 - Beginning 



3 - Moderate 

4 - Advanced 

5 - Highly advanced 



VIII. TECH PREP SEQUENCE OF STUDY 

School District: Coordinated sequence of vocational courses 
School District: Applied Academics, infused or discreet 
High School and/or District Focus on Career Paths or Clusters 
MSUCOT-GF: Coordinated sequence of vocational courses 
MSUCOT-GF: Applied Academics, infused or discreet 

IX. ORGANIZATION/ADMINISTRATION OF TECH PREP CONSORTIUM 

Effectiveness of structure and use of committees/ work groups 

Effectiveness of Tech Prep Coordinator 

Responsiveness of Tech Prep Staff 

Adequacy of Tech Prep funding 

Appropriate use of Tech Prep funds 

X. MARKETING OF TECH PREP 

CMTPC Staff: To Internal Market (students, teachers, parents) 

CMTPC Staff: To External Market (community & employers) 
MSUCOT-GF: To Internal Market (students, faculty) 

MSUCOT-GF: To External Market (community & employers) 

Local High School: To Internal Market 

Local High School: To External Market 

Business and Community Representatives: To all publics 

State Education Agencies (OPI, OCHE, MCVE): To all publics 

XI. OUTREACH/SUPPORT SERVICES FOR TARGET POPULATIONS 

Gender Equity 
Special Needs 

XII. SYSTEM TO MONITOR RESULTS / COLLECT DATA / EVALUATE EFFORT 

Central Montana Tech Prep Consortium 
Local High School & District 



0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 



0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 



0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 
0 1 2 3 4 5 



0 1 2 3 4 5 
0 1 2 3 4 5 



0 1 2 3 4 5 
0 1 2 3 4 5 



XIIL SYSTEM FOR ANNUAL REVIEW & REVISIONS OF OBJECTIVES 
Central Montana Tech Prep Consortium 



0 1 2 3 4 5 



Please make any additional comments on the organizational and programmatic components and the value and effectiveness of the Central 
Montana Tech Prep Consortium efforts. 



Permission to reprint material obtained from the Tech Prep Coordinator, Central Tech Prep 
Consortium, MT. 
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APPENDIX I 
Student Outcomes Rubric 



Rating for evaluating each component: 

Read each component and the rating criteria. 
Select the best rating based upon the current 
activity level in your consortium. 



E Exceptional Effort 
A Above Average Effort 
M Minimal Effort 
N No Effort 



Evaluative Criteria 


Component 


Data Source 


Rating 

E A M N 


Student Outcomes 


1 . Tech Prep consortium 
collects data on 
secondary and 
postsecondary Tech 
Prep student 
outcomes. 








2. Secondary Tech Prep 
student academic 
achievement has: 

a) decreased % 

b) remained the same 

c) increased % 








3. Postsecondary Tech 
Prep student 
academic 
achievement has: 

a) decreased % 

b) remained the same 

c) increased % 







Permission to reprint obtained from S. Ruhland, University of Minnesota, St. Paul, MN. 
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Evaluative Criteria 


Component 


Data Source 


Ratins 

E A M N 


Student Outcomes 


5. Postsecondary Tech 
Prep graduation rate 
has: 

a) decreased % 

b) remained the same 

c) increased % 








6. Tech Prep student 
entry into and 
retention in 
postsecondary 
education, advanced 
training, 

employment, and/or 
military has: 

a) decreased % 

b) remained the same 

c) increased % 








,7. Data is collected 
using multiple data 
collection strategies 
to include, but not 
limited to: surveys, 
interviews, 
observations, student 
transcript review, 
and follow-up 
studies (to include 
longitudinal data 
collection). 
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Program Outcomes Rubric 



Rating for evaluating each component: 

Read each component and the rating criteria. 
Select the best rating based upon the current 
activity level in your consortium. 



E Exceptional Effort 
A Above Average Effort 
M Minimal Effort 
N No Effort 



Evaluative Criteria 


Component 


Data Source 


Ratine 

E A M . N 


Program Outcomes 


1. Tech Prep 

stakeholders seek 
input and active 
involvement from 
community members 
to prepare all 
students, K- 1 6 and 
beyond for college 
and careers. 








2. Professional 
development 
activities (workshops, 
conferences, 
seminars, and 
internships) are 
provided to secondary 
and postsecondary 
staff (including 
teachers, counselors, 
and administrators) to 
support sustained and 
innovative 
experiences. 








3. The Tech Prep 

consortium promotes 
equal access for all 
students regardless of 
background or ability 
level. 
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Evaluative Criteria 


ComDonent 


Data Source 


Rating 

E A M N 


Program Outcomes 


4. Curriculum and 
instructional 
strategies (e.g., 
contextual learning, 
authentic assessment, 
integration, applied 
academics, and work- 
based learning) 
enable all students to 
achieve high 
academic and 
technical skills 
standards. 








5. Career guidance is 
provided for students 
to develop individual 
education/career 
plans (e.g., career 
planning guides, 
career pathways, 
career clusters). 








6. Articulation 
agreements are 
developed with 
secondary and two- 
and four year 
institutions and are 
reviewed on a regular 
basis. 








7. Tech Prep builds 
partnerships with 
secondary, and two- 
and four year 
institutions. 








8. Information is 
communicated to all 
stakeholders. 
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APPENDIX J 

Faculty/Administrator Questionnaire (FL) 
Orange • Osceola • Valencia Tech Prep Consortium 



Please indicate to what extent you agree or disagree with the following statements, 

0 = No opinion 1 = Strongly Disagree 2 = Disagree 3 = Agree 4 = Strongly Agree 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



0 12 3 4 



Teachers perceive that higher expectations have been established for 
students enrolled in a Tech Prep program of study. 

Tech Prep students are taking higher level academic courses in 
mathematics, science, and communications than did general 
education students in previous years. 

Teachers have positive views about students' ability to achieve higher 
expectations. 

Vocational- technical teachers stress their mission of preparing students 
for employment and further education. 

Academic teachers stress their mission of preparing students for 
employment and further education. 

Participating schools in the consortium have a special admissions process 
to Tech Prep that makes students feel distinct. 

More students take at least two mathematics and two science courses 
above basic or general courses. 

There is an increase in time and emphasis devoted to competency 
instruction. 

Academic instructional materials have been developed/adopted by 
academic teachers to emphasize mathematics, science, and communications 
skills as they relate to occupational fields or clusters. 

Vocational materials have been developed/adopted by vocational-technical 
teachers to emphasize mathematics, science, and communications skills 
as they relate to occupational fields or clusters. 

Instructional strategies are being used that stress reading and writing in the 
technical field. 

Increased number of joint planning sessions among vocational-technical 
and academic teachers. 

Increased number of demonstrated integrated teaching activities among 
vocational- technical and academic teachers at the secondary level. 
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0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 
0 12 3 4 

0 12 3 4 

0 12 3 4 

0 12 3 4 

0 12 3 4 

0 12 3 4 
0 12 3 4 

0 12 3 4 

0 12 3 4 
0 12 3 4 

0 12 3 4 

0 12 3 4 

0 12 3 4 



Increased number of demonstrated integrated teaching activities among 
vocational-technical and academic teachers at the postsecondarv level. 

Secondary and postsecondary teachers meet to coordinate curriculum 
development activities. 

Individualized educational plans for Tech Prep students are developed 
and used. 

Individualized Tech Prep student educational plans reveal that student's 
programs of study match their perceived goals. 

Individualized Tech Prep student educational plans reveal that students are 
taking a sequence of mathematics, science, and communications courses 
that is appropriate for their chosen occupational or technical field. 

All students are given an aptitude and occupational and/or technical 
assessment test and are assisted in interpreting the results. 

Counselors demonstrate an understanding of Tech Prep and are recruiting 
students. 

School-Based Tech Prep Coordinators assist students with career planning 
and Tech Prep assessments. 

Minimum reading and mathematics achievement levels are established to 
identify students needing extra help. 

Tech Prep admission process identifies students needing special help. 

Individualized educational plans are on file for students with basic skills 
deficiencies. 

Teachers and students verify coordination of vocational-technical, 
mathematics, science, and communications instruction for students with 
deficiencies. 

Special instruction is provided to students with mathematics, science, or 
communications deficiencies. 

Recent staff development or college credit instruction has been provided in 
applied teaching strategies for mathematics, science, and communications 
skills underlying the vocational-technical field. 

Curriculum and instructional plans and materials reflect mathematics, 
science, and communication competencies that underlie the vocational- 
technical field. 

Classroom observation reveals that vocational-technical teachers teach and 
reinforce mathematics, science, and communications competencies that 
underlie their vocational-technical field. 

Classroom observation reveals that academic teachers teach and reinforce 
academic competencies as they relate to appropriate occupational fields. 



162 



National Research Center for Career and Technical Education 



IS 7 



Measuring Tech Prep Excellence: A Practitioner's Guide to Evaluation 




0 



2 3 4 



Vocational-technical and academic faculty are given extended periods of 
time to learn from each other. 



0 12 3 4 
0 12 3 4 

0 12 3 4 

0 12 3 4 
0 12 3 4 

0 12 3 4 
0 12 3 4 




0 12 3 4 



0 12 3 4 



Vocational- technical and academic faculty share joint professional 
development activities. 

Students and teachers perceive that more time is being devoted to applied 
teaching and more real-life examples are used to associate competencies 
with vocational-technical applications. 

Vocational-technical and academic teachers have received recent staff 
development on applied instructional methods, effective teaching, and 
application of competencies. 

Curriculum and instructional plans reflect applied teaching strategies. 

Classroom observation reveals increased use of applied teaching methods 
for students following a Tech Prep program of study. 

Student handbooks and/or other student materials at the secondary and 
post-secondary level contain information on offered Tech Prep programs of 
study offered by the consortium. 

Regional occupational forecasts at the time the students enter Tech Prep 
indicate that above-minimum wage jobs in their field of study will be 
available. 

Businesses and industries participate in the periodic review and up-dating 
of programs of study and curricula to ensure the occupational/technical 
material taught is relevant for the current job market. 

Businesses and industries are actively sought to provide internships/ 
apprenticeships/mentors to students to give them experience in their 
field prior to placement. 



Permission to reprint material obtained from the 
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Post Secondary Administrator Survey of the Tech Prep Program (NY) 



As part of an effort to evaluate the New York State’s Tech Prep programs we are collecting perceptions 
about the effectiveness of the program. As an administrator involved in this program, your input is very 
important to us. Please read each of the following questions and provide us with your response. To 
maintain confidentiality of your responses, please place the survey in the accompanying envelope, seal it 
and return the envelope to . 



About You 

Gender: 1. Male 2. Female Title: 

Years of experience as an administrator . 

How long have you been involved with Tech Prep as an administrator? 



Support for Tech Prey 

1. The following are areas in which you may have received support for implementation of Tech Prep. For each, 



please circle the letter indicating the level of support you have received. 

Outstanding Sufficient 

1. Supplies, materials and equipment 1 2 


Insufficient 

3 


No SuDDort 
4 


2. Staff development. 


1 


2 


3 


4 


3. Planning time. 


1 


2 


3 


4 


4. Support from other teachers/faculty. 


1 


2 


3 


4 


5. Support from counselors/admissions staff. 


1 


2 


3 


4 


6. Support from administrators. 


1 


2 


3 


4 


7. Support from the business community. 


1 


2 


3 


4 



Staff Development 



2. The following is a list of staff development topics that address Tech- Prep related issues, for each, please indicate 
if you have attended or how you feel about attending them. (Please circle your responses.) 

Have Will Would like No need 

attended attend to attend to attend 


1 . General information about Tech Prep 

2. Recruitment, placement, and retention of 


1 


2 


3 


4 


students for Tech Prep 


1 


2 


3 


4 


3. School relationship with business/industry /labor 


1 


2 


3 


4 


4. School to work information 


1 


2 


3 


4 


5. Integrating vocational and academic content 

6. Promoting cooperation between secondary and 


1 


2 


3 


4 


post- secondary staff 


1 


2 


3 


4 


7. Job placement assistance for students 

Your Involvement in Tech Prep 


1 


2 


3 


4 



3. What has been your involvement in the planning and implementation of Tech Prep during the last three years? 
(Please check as many as apply.) 



1 . Served on the planning committee for Tech Prep. 

2. Helped develop Tech Prep curriculum. 

3. Helped develop the content of the articulation 
agreements between high school and community 
colleges. 

4. Advised students on Tech Prep recruitment and 
options. 

5. Visited other Tech Prep programs. 



6. Participated in Tech Prep meetings with business 
or community leaders. 

7. Attended national or regional conference on Tech 
Prep. 

8. Conducted workshops/training on Tech Prep. 

9. Presented material at national Tech Prep 
conferences. 

10. Presented material at national Tech Prep 
conference. 

11. Other 
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About Tech Prey 

4. The following areas reflect attitudes or perceptions about the Tech Prep program. For each, please circle your 
responses to the following statements using this scale: 



1= Strongly agree 2= Agree 3 = Slightly agree 4= Slightly disagree 5= Disagree 6= Strongly disagree 



1. Tech Prep is beneficial for former “general program” students. 

2. Tech Prep is beneficial for “regents program” students. 

3. Integrating academic and vocational skills is crucial to Tech 
Prep success 

4. Tech Prep is just another short-lived educational innovation. 

5. Tech Prep helps to orient students to current work place requirements. 

6. Tech Prep effectively prepares students for post secondary education 
and/or future employment. 

7. 1 support the continuation of the Tech Prep Program. 

8. I believe that the Tech Prep program helps retain students who may 
have been at risk. 

9. Students have positive attitudes about their experience in the Tech Prep 
program. 

10. Student self esteem is raised through the Tech Prep program. 

1 1. Articulation agreements are important for Tech Prep success. 

12. If federal Tech Prep funds were discontinued, my Tech Prep program 
would have difficulty in continuing. 

13. Administrators are knowledgeable about Tech Prep of funding and 
curriculum. 

14. School board members are knowledgeable about Tech Prep of 
funding and curriculum. 



SA SD 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 




5. 



Please provide us with any comments, concerns, and issues you may have about Tech Prep. 



Permission to reprint material obtained from B. Flugman, Center for Advanced Study in 
Education, NY. This survey developed by Dianna L. Newman, University at Albany, State 
University of New York. 
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APPENDIX K 

Teacher Survey of the Tech Prep Program (NY) 



Secondary /Post Secondary 

As a part of an effort to evaluate the New York State's Tech Prep programs we are collecting perceptions 
about the effectiveness of the programs. As a teacher involved in this program, your input is very 
important to us. Please read each of the following questions and provide us with your response. To 
maintain confidentiality of your responses, please place the survey in the accompanying envelope, seal it 
and return the envelope to . 



About You 

Gender: 1. Male 2. Female 

Highest degree: Bachelors Masters Doctorate 

Advanced certificate in 

What courses do you teach? 

How long have you been a high school teacher? 

How long have you been a teacher involved in Tech Prep? 



Support for Tech Prey 

1. The following are areas in which you may have received support for the implementation of Tech Prep. For each, 
please circle the number indicating the level of support you have received. 



1 . Materials for the classroom. 

2. Staff development. 

3. Planning time. 

4. Support from other teachers/faculty. 

5. Support from counselors/admissions staff, 

6. Support from administrators. 

7. Support from the business community. 



Outstanding 

1 

1 

1 

1 

1 

1 

1 



Sufficient 

2 

2 

2 

2 

2 

2 

2 



Insufficient 

3 

3 

3 

3 

3 

3 

3 



No Support 
4 
4 
4 
4 
4 
4 
4 



Staff Development 

2. The following is a list of staff development topics that address Tech Prep related issues, for each, please indicate 
if you have attended and/or how you feel about attending them. (Please circle your responses.) 





Have Will 


Would like 


No need 




Attended attend 


to attend 


to attend 


General information about Tech Prep. 
Recruitment, placement, and retention of 


1 2 


3 


4 


Students for Tech Prep 
School relationship with business/ 


1 2 


3 


4 


industry /labor. 


1 2 


3 


4 


School to work information. 


1 2 


3 


4 


Integrating vocational and academic content. 
Application of mathematics, science, and/or 


1 2 


3 


4 


communications competencies to the work setting. 


1 2 


3 


4 


Curriculum development. 


1 2 


3 


4 
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Your involvement in Tech Prey 

3. What has been your involvement in the planning and implementation of Tech Prep during the last three years? 
(Please check as many as apply.) 

1 . Served on the planning committee for Tech Prep 7. 

2. Helped develop Tech Prep curriculum 8. 

3. Helped develop the content of the articulation 9. 

agreements between high school and community 

colleges 10. 

4. Advised students on Tech Prep recruitment and 

options 11. 

5. Taught Tech Prep (applied, integrated) course 12. 

6. Participated in Tech Prep meeting with business or 

community leaders 

About Tech Prey 

4. The following areas reflect attitudes or perceptions about the Tech Prep program. For each, please circle your 
responses to the following statements using this scale. 

1 = Strongly agree 2 = Agree 3 = Slightly agree 4 = Slightly disagree 5 = Disagree 6 = Strongly disagree 



Attended national or regional conference on Tech Prep 
Conducted workshops/training on Tech Prep 
Presented material at regional Tech Prep 
Conference 

Presented material at national Tech Prep 
Conference 

Visited Tech Prep programs in other areas 

Other 



1. Tech Prep is beneficial for former “general program” students. 

2. Tech Prep is beneficial for “regents program” students. 

3. Integrating academic and vocational skills is crucial to Tech Prep success. 

4. Tech Prep is just another short-lived educational innovation. 

5. Tech Prep helps to orient students to current workplace requirements. 

6. Tech Prep effectively prepares students for post secondary education and/or 
future employment. 

7. I support the continuation of the Tech Prep Program. 

8. I believe that the Tech Prep program helps retain students who may have been 
at risk. 

9. Students have positive attitudes about their experience in the Tech Prep program. 

10. Student self esteem is raised through the Tech Prep Program. 

11. I encourage students to take higher levels of math and science prior to the 
implementation of Tech Prep. 

12. Students seek more career guidance because of Tech Prep. 

13. Tech Prep has not changed vocational education. 

14. Tech Prep courses are not appropriate for college-bound students. 

15. Students in the Tech Program are more focused on career goals than are 
non-regents program students. 



Agreement Scale 
SA SD 

12 3 4 5 6 

12 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 



1 2 3 4 5 6 



5. How would you assess the transition of students in Tech Prep program from high school to a post secondary 
institution? (Please choose one.) 



1. Extremely difficult 2. Very difficult 3. Some problems 4. Little problems 5. No problem at all 



If they faced any difficulty, what were they? (Please check as many as apply.) 



1. Being away from home 

2. Doing assignments 

3. Finding their way around campus 

4. Getting to know faculty 

5. Selecting courses 

6. Making friends 



7. More responsibilities 

8. Using computers 

9. Adjusting to a new study routine 

10. Finding a job 

11. Other 



6. Please provide us with comments, concerns, and issues you may have about Tech Prep. 



Permission to reprint material obtained from B. Flugman, Center for Advanced Study in Education, NY. 
This survey developed by Dianna L. Newman, University at Albany, State University of New York. 
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APPENDIX L 

High School Guidance Counselor Survey of the Tech Prep Program (NY) 

As a part of an effort to evaluate the New York State's Tech Prep programs we are collecting perceptions 
about the effectiveness of the program. As a guidance counselor involved in this program, your input is 
very important to us. Please read each of the following questions and provide us with your response. To 
maintain confidentiality of your responses, please place the survey in the accompanying envelope, seal it 
and return the envelope to . 

About You 

1. Gender: 1. Male 2. Female 

2. Highest degree: Bachelors Masters Doctorate 

Advanced certificate in 

3. What grade level or levels do you advise? 

4. How many years experience do you have as a Guidance Counselor? 

5. How long have you been involved with Tech Prep as a Guidance Counselor? 

6. How many students do you advise? 

7. How do you select students to direct towards the Tech Program? 



Staff Development 

8. The following is a list of staff development activities and topics that address Tech-Prep related topics. 
Please indicate your desire for information by circling the response that best represents your need and 
whether you have attended and/or how you feel about attending. (Please circle your responses.) 



1. General information about Tech Prep. 

2. Recruitment, placement, and retention 
of students for Tech Prep 

3. School relationship with business/ 
industry /labor. 

4. School to work information 

5. Integrating vocational and academic 
content 

6. Application of mathematics, science, 
and/or communications competencies 
to the work setting. 

7. Career development counseling for 
students 

8. Job placement assistance for students 

9. Information on labor market trends 

1 0. Methods of promoting Tech Prep to 
various consumers. 

1 1 . Promoting cooperation between 
secondary and post-secondary staff. 



Have attended 
have enough 
Information 

1 

1 

1 

1 

1 



1 

1 

1 

1 

1 

1 



Have attended 
would like 
information 

2 

2 

2 

2 



2 

2 

2 

2 

2 

2 



Would like 
to attend/get 
information 

3 

3 

3 

3 

3 



3 

3 

3 

3 

3 

3 



No need to 

attend/get 

information 

4 

4 

4 

4 

4 



4 

4 

4 

4 

4 

4 
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Your Involvement in Tech Prep 

9. What has been your involvement in the planning and implementation of Tech Prep during the 
last three years? (Please check as many as apply.) 



1 . Served on the planning committee for Tech Prep. 

2. Helped develop Tech Prep curriculum. 

3. Helped develop the content of the articulation 

agreement between high school and community 
colleges. 

4. Advised students on Tech Prep recruitment and 

options. 

5. Visited Tech Prep programs in other areas. 

6. Participated in Tech Prep meetings with 

business or community leaders. 

About Tech Prey 

10. The following areas reflect attitudes or perceptions about the Tech Prep program. For each, please 
circle your responses to the following statements using this scale: 



7. Attended national or regional conference on Tech 
Prep. 

8. Conducted workshops/training on Tech Prep. 

9. Presented material at regional Tech Prep 
conference. 

10. Presented material at national Tech Prep 
conference. 

11. Other 



1 = Strongly agree 2 = Agree 3 = Slightly agree 4 = Slightly disagree 5 = Disagree 6 = Strongly disagree 



1. Tech Prep is beneficial for former “general program” students. 

2. Tech Prep is beneficial for “regents program” students. 

3. Integrating academic and vocational skills is crucial to Tech Prep success. 

4. Tech Prep is just another short-lived educational innovation. 

5. Tech Prep helps to orient students to current work place requirements. 

6. Tech Prep effectively prepares students for post secondary education and/or 
future employment. 

7. I support the continuation of the Tech Prep Program. 

8. I believe that the Tech Prep program helps retain students who may have been 
at risk. 

9. Students have positive attitudes about their experience in the Tech Prep program. 

10. Student self esteem is raised through the Tech Prep Program. 

11.1 encourage students to take higher levels of math and science prior to the 
implementation of Tech Prep. 

12. Students seek more career guidance because of Tech Prep. 

13. Tech Prep has not changed vocational education. 

14. Tech Prep courses are not appropriate for college-bound students. 

15. Students in the Tech Program are more focused on career goals than are 
non-regents program students. 



Agreement Scale 
SA SD 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 

1 2 3 4 5 6 



1 1 . Please provide us with any comments, concerns, and issues you may have about Tech Prep. 



Permission to reprint material obtained from B. Flugman, Center for Advanced Study in Education, NY. 
This survey developed by Dianna L. Newman, University at Albany, State University of New York 
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APPENDIX M 

Annual Self-Assessment of Student Performance (NY) 



Tech-Prep Consortium of Queens 



1. Your Name: 



Grade: Date: 



2 . 



3. 




a. 

b. 

c. 

d. 

e. 

/ 

g- 

h. 

i. 

j- 

k. 

l . 

m. 



4. 

5. 



Name of your High School: 



Circle as Follows: 

1 = no change 

2 - some change 

3 = good change 

Performance Areas 



Thinking (can think purposefully and consequentially, is fair-minded) 
Reading (can read purposefully and consequentially) 

Writing (can write purposefully and consequentially) 

Listening (can listen purposefully and consequentially) 

Speaking (can speak purposefully and consequentially) 

Problem-solving (you are able to identify problems, gather resources, 
identify activities, and consider consequences) 

Resources (identifies, organizes, plans, and allocates resources) 
Interpersonal (works with others) 

Information (acquires and uses information) 

Systems (understands complex interrelations) 

Technology (works with a variety of technologies) 

Entrepreneurial (takes risks by responsibly meeting situational demands) 
Other (specify) 



1 2 3 
1 2 3 
1 2 3 
1 2 3 
1 2 3 

1 2 3 
1 2 3 
1 2 3 
1 2 3 
1 2 3 
1 2 3 
1 2 3 
1 2 3 



Do the Tech-Prep courses you take help you to decide your future career path? Yes No 



What do you see as the difference between the whole, analytical approach to learning and the traditional 
approach to learning? 



6. Use this space for any comments or suggestions on any aspect of the Tech-Prep program: 



7. What is your overall rating of the Tech-Prep program taking everything into account (circle one): 
1 (Poor) 2 (Fair) 3 (Average) 4 (Good) 5 (Excellent) 



Permission to reprint material obtained from Mary Ann Meyer, Tech Prep Consortium of Queens, NY. 
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Graduate Exit Survey (OH) 



MIAMI VALLEY TECH PREP CONSORTIUM 

Congratulations on your recent graduation from Sinclair Community College. As a Tech Prep/Sinclair Associate degree graduate, you are indeed 
a pioneer. You have worked hard to achieve this milestone, and we are very proud of your accomplishments and dedication. 

Please take a few moments to complete this Graduate Exit Survey. It will help us (1) learn more about your experiences and (2) continue to 
improve the Tech Prep Program for students who will follow you. Thank you for your help. Please return this survey in the enclosed envelope. 

NAME: 

ADDRESS: 

CITY/STATE/ZIP: 

PHONE: Email: 

PERMANENT ADDRESS (parent or family member) 

NAME: ' 

ADDRESS: 

CITY/STATE/ZIP: 

PHONE: Email: 

TECH PREP PROGRAM: 

Electronics Engineering Technology Industrial Engineering Allied Health 

Automotive Technology Computer Support Environmental 

WORK EXPERIENCE DURING YOUR TECH PREP STUDIES AT SINCLAIR: 

Company: 

Position: 

WHAT ARE YOUR PLANS FOLLOWING GRADUATION?: 



Employment: Company: 

Position: 

Est. annual salary: 

Continue at Sinclair: Program: 

Continue at 4-year institution: Name of Institution: 

Planned major/program: 

Military: (Branch): 

Other: 

DID THE TECH PREP PROGRAM FULFILL ITS COMMITMENT TO YOU? 

Provide a “seamless” education with minimum repetition 

Provide a smooth transition from high school to Sinclair 

Provide a smooth transition from Sinclair to employment 

Related academic areas to the technology field you chose (Applied academics) 

Provided scholarship assistance as promised 

WERE YOU SATISFIED WITH THE TECH PREP PROGRAM? Very Somewhat Not at all 

WOULD YOU CHOOSE THE TECH PREP PATHWAY AGAIN? Yes No 

WHAT ASPECTS OF TECH PREP WERE MOST BENEFICIAL? 



WHAT ASPECTS OF TECH PREP WAS MOST DIFFICULT? 



HOW COULD THE TECH PREP PROGRAM BE IMPROVED OR EXPANDED? 



Miami Valley Tech Prep Consortium, Sinclair Community College, 444 W. Third Street, Dayton, OH 45402 

Permission to reprint material obtained from the Director, Miami Valley Tech Prep Consortium, OH. 
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APPENDIX N 

Parent/Guardian Perception Survey 



School District of Hillsborough County 
Hillsborough Community College 
TECH PREP Consortium 



I learned about TECH PREP through: 

A. Guidance Counselor/Occupational Specialist 

B. Materials sent to my home 

C. Another parent 

D. Open House 

E. Other: (Specify) 



Directions: Using the following scale, record your response in the left hand column the letter that 
represents your awareness of TECH PREP. 

A = YES B = NO C = I DON’T KNOW 




Are you aware that your child had signed up for Tech Prep? 

My child can earn college credit while in high school by completing a TECH PREP program. 

My child needs to take rigorous mathematics and science classes to be ready for higher education in a 
TECH PREP program. 

My child can qualify for the Gold Seal Scholarship by completing a TECH PREP program. 

With TECH PREP, my child has a variety of higher education options, such as university, community 
college, adult technical center or adult apprenticeship. 

TECH PREP can increase my child's earning ability upon graduation from high school. 

TECH PREP will prepare my child to enter the workplace and higher education. 

TECH PREP students take the same level of courses as university-bound students. 

TECH PREP will provide my child with employment skills as well as preparation for a higher education. 
By completing TECH PREP, my child will have a head start toward an Associate in Science degree. 



Thank you for your time. Please return the results in the enclosed stamped, self-addressed envelope. 



Permission to reprint material obtained from the Tech Prep Administrative Resource Teacher, Hillsborough Tech 
Prep Consortium, School District of Hillsborough County, FL. 
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APPENDIX O 
Employer Follow-up Survey 




School District of Hillsborough County 
Tampa, Florida 


Directions: 


On this survey, you will be asked to evaluate the current work skills of 

who was a TECH PREP graduate. Read each 


statement and record your response in the left hand column the letter that best reflects 
your opinion. 


A 


= Very Good B = Good C = Average D = Poor E = Does Not ApdIv 


This TECH PREP graduate can: 


1 . 


Identify, organize, plan, and allocate resources. 


2. 


Work with others. 


3. 


Acquire and use information. 


4. 


Follow policies and procedures effectively. 


5. 


Work within system in which your company operates. 


6. 


Work with a variety of tools or equipment, including computers. 


7. 


Locate, understand, and interpret written information. 


8. 


Communicate thoughts, ideas, information, and messages in writing. 


9. 


Perform basic computations. 


10. 


Interpret and respond to verbal messages or directions. 


11. 


Organize ideas and communicate orally. 


12. 


Generate new ideas, solve problems. 


13. 


Display responsibility, self-esteem, sociability, self-management, and 
integrity and honesty. 


Please provide any additional comments you may have to assist us with evaluating the Tech Prep 
initiatives. Thank you again for your assistance. 


Permission to reprint material obtained from the Tech Prep Administrative Resource Teacher, 
Hillsborough Tech Prep Consortium, School District of Hillsborough County, FL. 
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APPENDIX P 

Education to Careers Follow - Up Survey 



Part One 

High School Experiences 

This set of questions asks about the education and jobs you may have had during high school. 

1 . How useful was what you learned in high school to what you've done since high school 
graduation? (Circle the one best response.) 

1 Extremely useful 

2 Very useful 

3 Fairly useful 

4 Somewhat useful 

5 Not at all useful 

2. While you were enrolled in high school, did you ever hold a job? 

1 no SKIP TO QUESTION 5 ON PAGE 2 

2 yes CONTINUE TO QUESTION 3 

3. Please estimate the amount of money you made per hour in the last job you held 
before high school graduation. If you held more than one job, choose the job where 
you worked the most hours. (Circle the one best response.) 

1 Zero.... My job was unpaid 

2 Less than $5.25 per hour 

3 $5.26 to $6.00 per hour 

4 $6.01 to $7.00 per hour 

5 $7.0 1 to $8.00 per hour 

6 More than $8.00 per hour 

7 I don't know 

4. In the last job you held before high school graduation, how many total hours did you work 
during a typical week? (Circle the one best response.) 

1 Less than 5 hours 

2 6-10 hours 

3 11-20 hours 

4 21-30 hours 

5 31-40 hours 

6 More than 40 hours 
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5. Sometimes high schools arrange special opportunities for students to leam about careers. 

Which of the following experiences did you participate in while enrolled in high school ? 

(Circle all that apply.) 

1 Job shadowing - where students spend time following workers in a work site 

2 Internship - where students work for an employer to leam about a particular 
occupation or industry 

3 Cooperative education (Co-op) - where students work in a job for which they get 
school credit 

4 Tech Prep - where students participate in academic and vocational classes that 
have a clear career focus that links high school and two-year college 

5 Youth apprenticeship - where students participate in a guided worksite learning 
experience that is closely associated with their classes in high school and two-year 
college 

6 School-sponsored enterprise/business — where students operate a business or 
provide services as part of an enterprise sponsored by a school and often located on 
school property 

7 Career Academy - where groups of students and teachers stay together for some of 
their classes in high school, students take classes in an occupational area and have 
jobs related to this occupation 

8 Community service and service learning - where students do volunteer work in 
the community that may or may not be related to their career interests 

9. I did not participate in any of the above experiences while in high school 

6. Since graduating from high school have you ever enrolled in any college or university? 

( Circle the one best response.) 

1 no SKIP TO QUESTION 17 ON PAGE 5 

2 yes CONTINUE TO QUESTION 7 
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Part Two 

College Experiences 

This set of questions asks about the schools, colleges and universities that you have attended since high 
school graduation. 

7. Since graduating from high school, which of the following types of colleges or universities 
have you attended? (Circle all that apply.) 

1 Two-year community or junior college 

2 Vocational, technical, trade or business school 

3 Four-year college or university 

4 Other - Specify 

8. List the name of each college/university that you have attended, the location of the 
college/university by city and state, and the dates you started and ended enrollment. (If 
necessary, list additional colleges/universities on a separate sheet and return it with this 

survey.) 

College/University City/State 

Name 

EXAMPLE: Lincoln Lincoln, Illinois 

College 

9. How soon after high school graduation did you first enroll in a college/university? (Circle 
the one best response.) 

1 Within six months of high school graduation 

2 Six months to one year after high school graduation 

3 More than one year after high school graduation 

10. Of the following categories of college majors, which one most closely matches your chosen 
program of study? (Circle the one best response.) 

1 Agriculture (agricultural business, natural resources, animal science, 
horticulture, farm management) 

2 Allied health (medical, dental, occupational or physical therapy, nursing, 
radiology, veterinary science) 

3 Business and information technology (accounting, banking, computer 
programming, information processing, secretarial, general management, 
marketing, real estate, travel agent) 

4 Human services (teacher, teacher assistant, child care, fashion design, hotel 
management, chef) 

5 Humanities, Fine arts and Communications (art, drama, English, music, 
foreign languages, journalism, television/radio, commercial art) 



Dates started 
and ended 

Sept. 1, 1996 -May 15,1998 
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6 Industrial/engineering technology (construction, machining, electronics, 
automotive, manufacturing) 

7 Public service (law and law enforcement, legal assistant, firefighting, social 
worker, armed services) 

8 Science and mathematics (biology, chemistry, math, physics, engineering, 
computer science) 

9 Social and behavioral sciences (economics, history, psychology, sociology) 

1 0 Other - Specify 

1 1 I am undecided about my college major 

1 1 . Over all the time you have enrolled in college, have you usually been a full time or part time 
student? (Circle the one best response.) 

1 Full time (more than 12 credit hours per term/semester) 

2 Part time (less than 1 2 credit hours per term/semester) 

12. How much of your college program (as identified in Question 10 above) have you already 
completed ? (Circle the one best response.) 

1 Less than one-fourth 

2 About one-fourth 

3 About half 

4 About three-fourths 

5 More than three-fourths 

6 I have completed my entire college program 

13. Which of the following college credentials are you seeking right now ? (Circle the one best 
response.) 

1 A certificate or license requiring less than a two-year degree 

2 An associate of applied science degree in an occupational-technical field 

3 A transfer associate of science or arts degree designed for continuation at a four-year 
college 

4 A bachelor’s degree 

5 Other- Specify 

6 I am no longer seeking a college credential 
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14. Since high school graduation, which college credentials have you already received ? (Circle 
all that apply.) 

1 A certificate or license requiring less than a two-year degree 

2 An associate of applied science degree in an occupational-technical field 

3 A transfer associate of science or arts degree designed for continuation at a four-year 
college 

4 A bachelor's degree 

5 Other- Specify 

6 I have not completed any college credential since high school graduation 

15. If it were up to you, how far would you go in school? 

1 A certificate or license requiring less than a two-year degree 

2 An associate of applied science degree in an occupational-technical field 

3 A transfer associate of science or arts degree designed for continuation at a four-year 
college 

4 A bachelor's degree 

5 A graduate or advanced degree such as Master's, Doctorate or M.D. 

6 Other - Specify 

16. How confident do you feel that you will reach your ultimate educational goal? 

1 Extremely confident 

2 Very confident 

3 Fairly confident 

4 Somewhat confident 

5 Not at all confident 
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Part Three 

Employment after High School 

This set of questions asks about the jobs that you have had since you graduated from high school. 

17. Please indicate the number of jobs that you have had since graduating from high school. 
(Circle the one best response.) 

1 1 job 

2 2 jobs 

3 3 jobs 

4 4 jobs 

5 5 jobs or more 

6 I have not had any jobs since high school graduation. 

SKIP TO QUESTION 24 ON PAGE 7 

1 8. What is your employment status at the present time? (Circle the one best response.) 

1 Employed full-time (35 or more hours per week) 

2 Employed part-time (less than 35 hours per week) 

3 Unemployed, but actively seeking employment 

4 Unemployed and not seeking employment 

5 Serving in the military full-time 

6 Other - Specify 

19. Sometimes people have more than one job. How many jobs do you hold at the present time? 

1 0 jobs SKIP TO QUESTION 24 ON PAGE 7 

2 1 job 

3 2 jobs 

4 3 jobs or more 

20. Thinking of your primary job as the job that takes up the largest amount of time you spend 
working each week, how many months have you worked in your primary job ? (Circle the 
one best response.) 

1 Less than 6 months 

2 6 to 12 months 

3 1 3 to 24 months 

4 25 to 36 months 

5 36 months or more 
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21 . Please estimate the amount of money that you make per hour in your primary job . (Circle 
the one best response.) 

1 Zero. My primary job is unpaid 

2 Less than $5.25 per hour 

3 $5.26 to $6.00 per hour 

4 $6.01 to $7.00 per hour 

5 $7.01 to $8.00 per hour 

6 $8.01 to $9.00 per hour 

7 $9.01 to $10.00 per hour 

8 $10.01 to $1 1.00 per hour 

9 $1 1.01 to $12.00 per hour 

10 $12.01 to $13.00 per hour 

11 More than $13.00 per hour 

1 2 I don't know 

22. Which of the following best describes the type of job you have as your primary job ? 

(Circle the one best response.) 

1 Entry level or unskilled job - minimal training is required and little orientation is 
provided by employers. Hiring is usually not very competitive. 

2 Semi-skilled job - usually requires 6 months to 1 year of specific training, college 
education or equivalent skills and experiences prior to being hired. Hiring is usually 
competitive. 

3 Skilled or technical job - usually requires 1 year to 2 years of specific training or 
college education prior to being hired. Hiring is usually very competitive. 

4 Professional job - usually requires 2 to 4 years or more of specific training. College 
degrees and/or state professional licensure or certification are often required. Hiring 
is usually extremely competitive. 

23. How satisfied are you with your primary job ? (Circle the one best response.) 

1 Extremely satisfied 

2 Very satisfied 

3 Fairly satisfied 

4 Somewhat satisfied 

5 Not at all satisfied 
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24. Which of the following best describes the type of job you would ultimately like to get? 
(Circle the one best response.) 

1 Entry level or unskilled job - minimal training is required and little orientation is 
provided by employers. Hiring is usually not very competitive. 

2 Semi-skilled job - usually requires 6 months to 1 year of specific training, college 
education or equivalent skills and experiences prior to being hired. Hiring is usually 
competitive. 

3 Skilled or technical job - usually requires 1 year to 2 years of specific training or 
college education prior to being hired. Hiring is usually very competitive. 

4 Professional job - usually requires 2 to 4 years or more of specific training. College 
degrees and/or state professional licensure or certification are often required. Hiring 
is usually extremely competitive. 

25. How confident do you feel that you will reach your ultimate career goal? (Circle the one 
best response.) 

1 Extremely confident 

2 Very confident 

3 Fairly confident 

4 Somewhat confident 

4 Not at all confident 

Part Four 

Just a few more questions about you . . . 

26. What is your gender? 

1 Male 

2 Female 



27. Are you of Hispanic origin or descent? 

1 no 

2 yes 
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28. What is your racial/ethnic background? (Circle the one best response.) 

1 White 

2 Black or African American 

3 Asian or Pacific Islander 

4 American Indian or Alaska Native 

5 Other- Specify 

29. What is your current marital status? (Circle the one best response.) 

1 Single 

2 Single with children 

3 Married 

3 Married with children 

30. How much education have your parent(s) or guardian(s) completed? (Circle the one best 
response for each parent/guardian who you consider to have had the most influence on your 



life.) 

My father, stepfather or male 
guardian has completed: 

(Circle one response.) 



My mother, stepmother or female guardian 
has completed: 

(Circle one response.) 

1 Less than high school graduation 

2 High school graduation 

3 Some college but no degree 

4 Two-year Associate degree 

5 Four-year Bachelor's degree 

6 Graduate degree such as Master's, 



1 Less than high school graduation 



2 High school graduation 



3 Some college but no degree 



4 Two-year Associate degree 



5 Four-year Bachelor's degree 



6 Graduate degree such as Master's, 
Doctorate or M.D. 



Doctorate or M.D. 
7 I don't know 



7 I don't know 
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31. What was your approximate total annual family income while you were in high school and 

living with your parent(s)/ guardian(s)? (Circle the one best response.) 

1 $14,999 or less 

2 $15,000-$29,999 

3 $30,000-$44,999 

4 $45,000-$59,999 

5 $60,000-$74,999 

6 $75,000-$89,999 

7 $90,000 or more 

8 I don't know 

32. Where do you reside at the present time? (Circle the one best response.) 

1 I live with my parent(s)/guardian(s) 

2 I live alone 

3 I live with a spouse or significant other 

4 I live with a friend or roommate 

5 Other— Specify 

PLEASE RETURN THIS QUESTIONNAIRE IN THE POSTAGE-PAID ENVELOPE 
PROVIDED. WE THANK YOU FOR PARTICIPATING IN THIS STUDY. 

ID NO: 



Permission to reprint material obtained from D. Bragg, Project Director, University of Illinois 
Urbana/Champaign, IL. 
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APPENDIX Q 
OVAE Career Clusters 


Career Clusters 


Definitions 


1. Agriculture and 
Natural 
Resources 


Planning and managing agriculture, food, fiber, and natural resources 
systems. Production of agricultural commodities, including food, fiber, 
wood products, horticultural crops, and other plant and animal products. 
Financing, processing, and marketing and distribution of agricultural 
products; farm production and supply and service industries; 
horticulture and landscaping services, and the use and conservation of 
land and water resources; development and maintenance of recreational 
resources. It also includes mining and extraction operations and related 
environmental management services. 


2. Construction 


Designing, planning, managing, building, and maintaining physical 
structures and the larger built environment including roadways and 
bridges and industrial, commercial and residential facilities and 
buildings. 


3. Manufacturing 


Planning, managing and performing the processing of materials into 
intermediate or final products and related professional and technical 
support activities such as production planning and control, maintenance 
and manufacturing/process engineering. 


4. Transportation, 
Distribution and 
logistics Services 


Planning, management, and movement of people, materials, and goods 
by road, pipeline, air, rail and water and related professional and 
technical support services such as transportation infrastructure planning 
and management, logistics services, mobile equipment and facility 
maintenance. 


5. Information 
Technology 
Services 


Designing, developing, managing and operating communication and 
information technology systems, networks, and related hardware and 
software for telecommunications and computing services. 


6. Wholesale/Retail 
Sales and 
Services 


Planning, managing and performing wholesaling and retailing services 
and related marketing and distribution support services including 
merchandise/product management and promotion. 


7. Financial 
Services 


Planning, managing and providing banking, investment, financial 
planning, and insurance services. 


8. Hospitality and 
Tourism 


Planning, managing and providing lodging, food, recreation, convention 
and tourism, and related planning and support services such as travel- 
related services. 


9. Business and 
Administrative 
Services 


Planning, managing, and providing administrative support, information 
processing, accounting, and human resource management services and 
related management support services. 
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10. Health Services 


Planning, managing, and providing diagnostic, therapeutic, and 
information and environmental services in health care. 


1 1 . Human Services 


Planning, managing, and providing human services including social and 
related community services. 


12. Arts, Audio 
Video Tech, and 
Communications 
Services 


Designing, producing, exhibiting, performing, writing, and publishing 
multimedia content including visual and performing arts and design, 
journalism, and entertainment services. 


13. Legal and 
Protective 
Services 


Planning, managing and providing judicial, legal, and protective 
services including professional and technical support services in the fire 
protection and criminal justice systems. 


14. Scientific 
Research, 
Engineering and 
Technical 
Services 


Planning, managing, and providing scientific research and professional 
and technical services (e.g., physical science, social science, 
engineering) including laboratory and testing services, and research and 
development services. 


15. Education and 
Training 
Services 


Planning, managing and providing education and training services, and 
related learning support services including assessment and library and 
information services. 


16. Public 

Administration/ 

Government 

Services 


Planning, managing and providing government legislative and 
administrative and regulatory services and related general purpose 
government services at the federal, state and local levels. 



Permission to reprint material obtained from the U.S. Department of Education, OVAE, Washington, DC. 



188 



National Research Center for Career and Technical Education 



Measuring Tech Prep Excellence: A Practitioner's Guide to Evaluation 



16 Career Clusters and Examples of Occupations Found Within Each Cluster 



Agricultural & Natural Resources 

Food Scientist 
Environmental Engineer 
Agriculture Teacher 
Animal Scientist 
Biochemist 

Agricultural Sales Director 


Arts, A/V Technology & Communications 

Actor-Actress 
Video Producer 
Journalist 
Audio Engineer 

Tele-Communications Technologist 
Printing/Graphics Technologist 


Business and Administration 

Human Resource Administrator 
Administrative Specialist 
Financial Analyst 
International Trade Manager 
Entrepreneur 
Accountant 


Architecture and Construction 

Contractor 

Architect 

Electrician 

Heavy Equipment Operator 

Carpenter 

Plumber 


Education and Training 

Teacher 

Principal 

School Counselor 
College Professor 
Corporate Trainer 
Coach 


Finance 
Stock Broker 
Banker 

Insurance Agent 
Financial Planner 
Loan Officer 
Tax Examiner 


Health Science 
Pediatrician 
Physical Therapist 
Radiologic Technologist 
Occupational Therapist 
Medical Assistant 
Hospital Administrator 


Hospitality and Tourism 

Lodging Manager 
Chef 

Travel and Tourism Manager 
Food Service Manager 
Restaurant Manager 
Leisure and Entertainment Manager 
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Human Services 
Social Worker 
Psychologist 
Child Care Worker 
Substance Abuse Specialist 
Employment Specialist 
Psychotherapist 


Information Technology 

Software Engineer 
Network Administrator 
Web Designer/Developer 
Database Manager 
Technical Writer 
Multimedia Producer 


Law and Public Safetv 

Attorney 
Fire Fighter 
Police Officer 
Judge 
Paramedic 
Paralegal 


Manufacturing 

Machinist 

Manufacturing Engineer 
Automated Process Technician 
Production Engineer/Technician 
Welding Technician 
Quality Technician 


Government and Public Administration 

Legislator 
City Manager 
Policy/Budget Analyst 
Recreation/Parks Director 
State/Federal Agency Director 
Urban/Regional Planner 


Retail/Wholesale Sales and Service 

Sales Associate 
Interior Designer 
Marketing Director 
Buyer 

Real Estate Broker 

Customer Service Representative 


Scientific Research/Engineering 

Chemical Engineer 
Mathematician 
Bio Technologist 
Electrical Engineer 
Biologist 
Oceanographer 


Transportation, Distribution & Logistics 
Pilot 

Automotive Technician 
Logistics Manager 
Flight Attendant 
Warehouse Manager 
Truck Driver 
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APPENDIX R 

Tech Prep Database Programs 



Consortium Name: Manatee County Schools in Bradenton, FL 
Contact: Deborah Merrill, Career Education Specialist 

Address: School Board of Manatee County, PO Box 9069, Bradenton, FL 34206-9069 

Phone: 941-741-7298 

Email: Merrilld@fc.manatee.kl2.fl.us 

Description of Database Program: The Florida Department of Education maintains an 
Automated Student Information System that allows local school districts to enter data for 
individual students via computer. The information can be retrieved by the state for analysis and 
dissemination of statistics related to student enrollment and educational programs. When the 
Tech Prep education program began, a data element was put into place to identify Tech Prep 
participants. In 1 998, the Student Information System was modified to include data elements that 
would facilitate collecting information to determine the level of activity of certain School to 
Work activities. The state added data elements to identify participants in activities such as 
mentoring/coaching, school-based enterprise, academy enrollment, and service learning. 

The Manatee County school district added five additional elements to those the state initiated. 
The additional elements enabled district personnel to collect and analyze data related to (1) 
career assessment, (2) career-related classroom activities, (3) workplace tours, (4) job 
shadowing, and (5) the impact of curriculum integration. At the close of each school year, a 
spreadsheet is produced detailing the number of participants in each of the areas of activity. The 
information is analyzed and reported demographically for each high school in the district. The 
process of entering data is time consuming, but the information gained for evaluation purposes is 
invaluable. 

Permission to reprint material obtained from the Career Education Specialist, Manatee County 
Schools, FI. 



Consortium Name: Tech Prep Consortium Region 1-B 
Contact: Vickie Wagner, Tech Prep Specialist 

Address: Purdue University North Central, 1401 South US Highway 421, 

Westville, IN 46391-9528 
Telephone: 219-785-5362 
Email: vickiew@.purduenc.edu 

Description of Database Program: The Indiana Department of Education developed a 
statewide survey to address Perkins III accountability. (Federal dollars supported the 
development of this survey). Surveys were sent to every high school in the state, creating the 
baseline for FY00. In order to use the information, a database was created in Excel linked to a 
word document. These programs were utilized instead of Access as all regions had access to 
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these particular software programs. Regional Tech Prep specialists, administrators, educators, 
and the various state departments can then utilize the data for analysis and dissemination of 
information. Each high school will be able to see growth through comparisons between baseline 
FYOO and subsequent years, FY01, FY02, FY03, and FY04. Additionally, schools can compare 
their individual numbers to their region and to the state aggregate data. 

Categories within the database system include “All Seniors” and a subgroup, “All Senior 
Technical Preparation/Vocational Completers.” Specific data analysis can be obtained for 
number of (1) career plans on file; (2) students enrolled in classes that are articulated with a 
postsecondary partner; (3) students passing the ISTEP+ Graduation Qualifying Exam; (4) 
students completing 4 years of English that cover core 40 competencies; (5) students completing 
3 years of mathematics, at least 2 of which cover core 40 competencies; (6) students completing 
3 years of science, at least 2 of which cover core 40 competencies; (7) students completing or 
expecting to complete a job-shadowing experience during high school; (8) students completing 
or expecting to complete work-based learning during high school; and (9) students expecting to 
earn a Certificate of Technical Achievement during high school. 

Permission to reprint material obtained from Vickie Wagner and Patti Shutt, Tech Prep 
specialist, Purdue University North Central, IN. 



Consortium Name: Prince George's County, Maryland 

Contact: Dr. Rose Mary Swartwood, Prince George’s Community College 

Address: 301 Largo Rd, Largo, MD 20774 

Phone: 301-322-0699 

Email: swartwrm@pg.cc.md.us 

Description of Database Program: The Prince George's County Public Schools and Prince 
George’s Community each have their own student information databases for maintaining Tech 
Prep student records. However, in order to follow up and track Tech Prep students at the high 
school and continued on to the community college a mechanism was needed to facilitate this 
follow through and provide required statistical information for state and federal reporting. An 
application was designed to be completed by each student (and signed by their parent or 
guardian) enrolled in designated “gateway” courses for our nine designated Tech Prep education 
programs. These applications are then scanned into a Microsoft Access database program. 
Designated individuals at the public schools and community college maintain this information. 
These collaborative effort enables teachers and counselors necessary information including: (a) 
student guidance as they graduate from high school, (b) statistical information need for both high 
school and community college enrollment reports, and (c) projections as well as recruitment 
information for the community college. 

Permission to reprint material obtained from Dr. Rose Mary Swartwood, Prince George’s 
Community College, Largo, MD. 
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Consortium Name: Mid Rivers Tech Prep 
Contact: Dan Busekrus, Director 

Address: Pike/Lincoln Technical Center, Box 38, Vo-Tech Road, Eolia, MO 63344 

Phone: 941-741-7298 

Email: busekrus@.pltc.kl2.mo.us 

Description of Database Program: Microsoft Access is used to maintain a database program of 
educators, administrators, and business partners. Columns, as needed, are inserted to record data 
as to conferences attended, participants at meetings, and other consortium activities. This 
database is then used to assist with mail merges. The database system, although simple, has been 
Mid Rivers Tech Prep consortium's best way of collecting data, and keeping it simple has made 
it extremely user friendly. 

Permission to reprint material obtained from Dan Busekrus, Mid Rivers Tech Prep Consortium, 
MO. 



Consortium Name: Tech Prep Consortium of Queens, NY 
Contact: Mary Anne Meyer, Acting Program Director 

Address: Queens Community College, 222-05 56 th Ave., Room H-456, Bayside, NY 1 1364. 

Or via 20 Illinois Ave., Long Beach, NY 11561 
Phone: 718-631-6315 or 516-897-1819 
Email: MMever5625@aol.com 

Description of Database Program: Software was designed and developed by the consortium to 
reflect the data elements that need to be reported to the state. Reports are generated that detail 
population, performance, completion, and articulation of Tech-Prep students. Training has also 
been done in regard to design of forms and strategies for collection of data throughout the state. 

Permission to reprint material obtained from Mary Ann Meyer, Tech Prep Consortium of 
Queens, NY. 

Consortium Name: Tulsa Tech Prep, OK 
Contact: Dr. Tana Garrett, Vocational Director 

Address: Charles Page High School, PO Box 970, Sand Springs, OK 74063 

Phone: 918-241-1920 

Email: garrettt@,sandsprings.kl 2,ok.us 

Description of Database Program: Sand Springs, Oklahoma, uses a commercial database. 
Current data includes transcript, GPA, current enrollment, ethnicity, age, date of birth, parent 
information, address, phone, attendance, and discipline records. Access to information can be 
carefully controlled at each computer station on the network utilizing both numbers and 
passwords. Access to any particular database can be restricted to any user. The use of “query” 
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statements allows the user to tailor the retrieval of information to specific needs. The network 
can encompass multiple sites allowing systemwide access to information. Access the Web site at 
http://k 1 2.ncs.com/ . 

Permission to reprint material obtained from Vocational Director, Charles Page High School, 
OR. 



Consortium Name: Mt Hood Regional Educational Consortium, OR 

Contact: Dr. Dan Walleri, Director of Research and Planning 

Address: Mt Hood Community College, 26000 SE Stark St, Gresham, OR 97030 

Phone: 503-491-6924 

Email: wallerid@mhcc.cc.or.us 

Description of Database Program: Tech Prep student data are downloaded from the college's 
central student records system and then loaded into Access at the desktop level. Access is used 
for database management and summary analysis. SPSS for Windows is used for statistical tests. 
The final report is prepared using Microsoft Word. 

Permission to reprint material obtained from Director of Research and Planning, Mt. Hood 
Community College, OR. 



Consortium Name: Western Wisconsin Business-School Consortium, WI. 

Contact Name: Elizabeth J. Brendel, Tech Prep Curriculum Specialist 

Address: Western Wisconsin Technical College, 304 Sixth Street N, PO Box C-0908, 

La Crosse, WI 54602-0908 

Phone: 608-785-9588 

Email: brendele@, western. tec . wi .us 

Description of Database Program: The Western Wisconsin Technical College in La Crosse, 
Wisconsin, uses Microsoft Word Access to collect Tech Prep data. The college puts this database 
on the Web as a communication tool to secondary instructors, counselors, and others regarding 
agreements, etc. The institution transfers information about students to its enrollment specialist 
and counselors internally. The college uses PeopleSoft to pull reports from its Client Reporting 
System of students after they leave high school and matriculate in the postsecondary system. 
Western Wisconsin Technical College is currently transitioning into PeopleSoft from an internal 
system called Super. The Tech Prep specialists call on their Client Reporting department to 
request a report. Final reports are created using Microsoft Word. 

Permission to reprint material obtained from Tech Prep Curriculum Specialist, Western 
Wisconsin Technical College, WI. 
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